wiki:access/APS1G_rmsdifferences

RMS differences from a set of APS1 G runs

Ilia ran the following 4 cases

  • solar2:/g/sc/flush/iliab/um_run_scripts/7.5.480 - job is running using Solar executable built with "-O3" on a 20x24 decomposition

  • solar2:/g/sc/flush/iliab/um_run_script/7.5.192 - job is running using the same Solar executable as above on a 12x16 decomposition

  • ngamai:/g/sc/fs4/flush/iliab/um_run_scripts/7.5.O2 - run completed on Ngamai using an executable built with "-O2"

  • ngamai:/g/sc/fs4/flush/iliab/um_run_scripts/7.5.O3 - run completed on Ngamai using an executable built with "-xavx -O3"

Here we treat the first case as truth and calculate RMS differences of all other cases relative to this. The "pa" files from this run have 3 hourly fields from a number of variables including MSLP. The packing means that the MSLP is only accurate to 1 Pa, but that doesn't affect the results here. Ideally we'd have hourly or higher frequency data for the first day.

This first graph shows the RMS differences of the 3 runs relative to the base case and also the difference between the -O3 and -O2 on ngamai. All cases are similar, suggesting that the compiler changes are similar in magnitude to the roundoff level differences due to changing the decomposition.

The second graph adds estimates of the RMS forecast error for the last90 days for ACCESS G1 and ECMWF from http://web.bom.gov.au/nob/nmoc/verify/verify_charts/daily_plots/verf_plot.html. This doesn't have global values, so I've used a weighted RMS average of the S and N annulus and the tropics. To about day 5 the forecast errors are an order of magnitude larger than the compiler and decomposition effects.

3D temperature differences

I modified the jobs to save model level temperature every step for the first 3 hours at full 64 bit precision. Changing decomposition gives a difference of around 3e-14 K on the first step, jumping to 1.5e-4 on the second (presumably when the initial changes due to the dynamics feed back in the physics). However the first step difference between ngamai (with v12 compiler) and solar (with v 11 compiler) is 1e-4, while the effect of O3 vs O2 on ngamai is similar to the effect of decomposition (2e-14). For comparison I also did a run with the biogenic aerosol set to zero as an indication of the possible effect of a configuration error. Over the first few hours this is clearly larger than the other differences, suggesting that this analysis can pick up at least some differences.

After 1 day the difference in all cases (including the missing aerosol) is around 0.1 K. This means the growth rate of roundoff level differences is fast enough to match real differences quite quickly. By comparison, Rosinski and Williamson (1997) http://dx.doi.org/10.1137/S1064827594275534 found that initial roundoff level differences with NCAR CCM2 grew much more slowly and only reached a temperature difference of 1e-9K after 1 day.

TODO

The first step differences on ngamai are larger than expected. Need runs with the v12 compiler on solar to be sure what's causing this.

Last modified 6 years ago Last modified on Oct 10, 2014 2:25:24 PM

Attachments (4)

Download all attachments as: .zip