APS1 ACCESS-C Experiments

Wenming LU lwenming@bom.gov.au, Michael Naughton mjn@bom.gov.au

ACCESS-C Background

ACCESS City model, ACCESS-C for short, is one of the ACCESS NWP systems operational in the Australia Bureau of Meteorology.
There are in ACCESS-C six pre-defined domains namely, BN, SY, VT, AD, PH, DN, which cover main cities, Brisbane, Sydney,
Melbourne-Hobart, Adelaide, Perth and Darwin. The general information of these domains is listed in the following table,

City	Domain	Lat. Range	Lon. Range	Grids	Res.	fcst_hrs
Brisbane	BN	-31.00S, -22.04S	148.00E, 156.03E	250x224	4km	36
Sydney	SY	-38.00S, -29.98S	147.00E, 155.03E	224x224	4km	36
VicTas	VT	-46.00S, -33.04S	139.00E, 150.99E	362x334	4km	36
Adelaide	AD	-39.50S, -29.53S	132.00E, 141.98E	278x306	4km	36
Perth	PH	-37.00S, -28.04S	112.00E, 122.98E	250x306	4km	36
Darwin	DN	-16.00S, -7.97S	128.50E, 139.98E	224x320	4km	36

APS1 ACCESS-C UMUI jobs

APS1 ACCESS-C is UM VN7.6 UMUI jobs. There are three sets of UMUI jobs for each domain,

• A: standard UMUI jobs
• B: number of convection calls per physical timestep is increased to 4 instead of 1 in the standard UMUI jobs
• C: half timestep of that in the standard UMUI jobs

City	Domain	A	B	C
Brisbane	BN	xbcjb	xbeab	xbeeb
Sydney	SY	xbcjs	xbeas	xbees
VicTas	VT	xbcjv	xbeav	xbeev
Adelaide	AD	xbcja	xbeaa	xbeea
Perth	PH	xbcjp	xbeap	xbeep
Darwin	DN	xbcjd	xbead	xbeed

These UMUI components are located on vayu:/g/data/access/AccessModelExperimentLibrary/aps1_access_c/beans/um_components.

For reference, the resources needed by the forecast step for each domain are listed below,

City	Domain	Num. of Cores	Wall Time (min)	CPU time (min)	Input Data Size(B)	Output Data Size(GB)
Brisbane	BN	100	28 (22*)	2756	294M/3.9G	16.0
Sydney	SY	100	25 (20*)	2420	272M/3.5G	15.0
VicTas	VT	100	52 (38*)	4760	620M/5.5G	33.0
Adelaide	AD	100	38 (28*)	3375	438M/4.9G	23.0
Perth	PH	100	37 (27*)	3605	396M/4.7G	16.0
Darwin	DN	100	28 (25*)	2480	309M/4.1G	18.0

• Note*: Wall time with 196 cores used.
• In the column of Input Data Size(B), the sizes of initial condition and lateral boundary condition files (IC/LBC) are listed.

Sample Input Data and Forecast Output Data

There are some sample data for the experimental run (20130204 00Z) of the APS1 ACCESS-C on vayu.

• Input data:
  • Dump file for producing initial condition: /g/data/access/AccessModelExperimentLibrary/aps1_access_c/data/input_data/iau0-dump.201302040000
  • Pi files for producing LBC: /g/data/access/AccessModelExperimentLibrary/aps1_access_c/data/input_data/qwxbeta_pi[009-040]_2013020400_utc_fc.um
  • Please note that those data will not be used when running the standard domains;
    the sample initial condition (IC) and lateral boundary condition (LBC) have been provided (see the Output Forecast Data);
    for other domains, um_reconfig and makebc jobs are needed to create the IC and LBC files
• Output Forecast Data
  • BN: /g/data/access/AccessModelExperimentLibrary/aps1_access_c/data/output_data/BN, including IC, LBC and forecast data
  • SY: /g/data/access/AccessModelExperimentLibrary/aps1_access_c/data/output_data/SY, including IC, LBC
  • VT: /g/data/access/AccessModelExperimentLibrary/aps1_access_c/data/output_data/VT, including IC, LBC
  • AD: /g/data/access/AccessModelExperimentLibrary/aps1_access_c/data/output_data/AD, including IC, LBC
  • PH: /g/data/access/AccessModelExperimentLibrary/aps1_access_c/data/output_data/PH, including IC, LBC
  • DN: /g/data/access/AccessModelExperimentLibrary/aps1_access_c/data/output_data/DN, including IC, LBC

UM Run Script

We provide a script access_c_um.sh for running these cases, located in the scripts directory

vayu:/g/data/access/AccessModelExperimentLibrary/aps1_access_c/scripts

To try a test run, please copy run script to your own place and change the following items if you need to

• 2nd line, $PROJECT to your valid project, such as dp9, du7 etc
• line 29, export MY_PFX=BN #change BN to AD SY VT DN PH as you wish
• line 30, export UMUI_ID=xbcjb #leave as is, or change xbcjb to another UMUI job id from table above of UMUI jobs

Next, submit the run script:

qsub access_c_um.sh

The forecast output will be stored at /short/$PROJECT/$USER/aps1_access_c/$DOMAIN given $DOMAIN is
one of BN AD SY VT DN PH you specify. For BN, the sample forecast data are store at
/g/data/access/AccessModelExperimentLibrary/aps1_access_c/data/output_data/BN for your comparison.

Temorary Note: UM Run Script on raijin

The APS1 ACCESS-C is under working process to be ported to raijin. The folder vayu:/g/data/access/AccessModelExperimentLibrary/aps1_access_c/

is rsynced to raijin:/short/dp9/wml548/aps1_access_c/. The executable for the UM forecasting and reconfiguration has been compiled

on raijin and tested fine.

raijin:/short/dp9/wml548/aps1_access_c/scripts/access_c_um.sh has also been updated for raijin. This script has been tested on BN domain

and the forecast data are located in raijin:/short/dp9/wml548/aps1_access_c/BN. Please refer to the previous section for the modifications

to make your own experiments.

Full ACCESS-C Run by CYLC

For researchers who need to be able to control the full set of steps of the ACCESS-C system, for example to run other dates or

to cycle through periods of running, the CYLC version of the APS1 ACCESS-C is provided. The steps to copy and run the CYLC

version are as follows.

1. First, run

/g/data/access/AccessModelExperimentLibrary/aps1_access_c/scripts/install_aps1_access_c_cylc.ksh

If the script runs successfully, it will create the following two directories in your home directory,

• aps1_access_c: contains binaries, scripts, configurations, environment files and log files
• cylc/aps1_access_c: CYLC suite definition and CYLC related scripts and configurations

2. To run CYLC requires loading necessary modules

module use /g/data/access/modules
module load cylc

*Note: Please delete any already loaded modules which conflict with this CYLC module.

3. Register the suite in CYLC scheduler,

cylc db register aps1_access_c $HOME/cylc/aps1_access_c

where aps1_access_c is the name of the suite in CYLC, $HOME/cylc/aps1_access_c is the PATH to the suite definition file.

4. The next step is to run the suite:

gcylc aps1_access_c

You should be able to see the following window

Please click the run button and type the required dates as in the following box (for this example run, input data is supplied for the 20130204 00Z case)

Finally the GUI control of the ACCESS-C system pops up as follows

The following section describes the steps in the full ACCESS-C CYLC sample suite.

Steps in the ACCESS-C CYLC Suite

ACCESS-C is a relatively simple forecast-only system compared with the Bureau Global and Regional operational assimilation and

forecast systems. We use a tree-like structure to display all tasks and descriptions. Note that tasks are listed in logically

running order, which is not the case in the (current) CYLC displays which use alphabetical order at each level of the tree.

• -->setup
  • AC_setup : setup the folders for all jobs; Please go to DATADIR=/short/$PROJECT$USER/aps1_access_c_S to check the output
  • wait_for: triggered by completion of AC_setup; check the availability of the input data for creating initial and LB conditions
  • AC_get_initial_conditions: triggered by wait_for; get the dump file
  • AC_get_LB_conditions: triggered by wait_for; get the those pi file

• --> brisbane
  • make_lbc: producing the lateral boundary condition
  • recon: reconfiguration step to produce the initial condition at ACCESS-C resolution
  • recon_lores: reconfiguration step to produce the initial condition at the original resolution (usually lower than ACCESS-C resolution)
  • finalise_recon: copy wind fields from the IC from step recon_lores to that from recon
  • um_run: UM forecasting job; equivalent for what access_c_um.sh does
  • um2grb: convert forecast data from PP format to grib as requested
  • ncdf: convert forecast data from PP format to netCDF as requested
  • opendap: archive forecast data to OPeNDAP folder for easy access
• --> sydney
• --> victas
• --> adelaide
• --> perth
• --> darwin

• -->finalise
  • AC_cleanup: clean up unnecessary files after forecasts finish

*Note: Jobs under each domain are same, i.e. as shown for brisbane domain.

Sample Plots

BN 20130204 00Z time+2 Forecast: Mslp_Precip(1 Hour); time+1 Forecast: Sfc Temp, Sfc Dewpoint, Sfc Wind

• Note: For all plots please click the image for display in the full size.

Sample Plots For Extreme Weather Cases

VT 20130321 00Z time+1 to time+6 Forecast: Mslp_Precip(1 Hour)

Oswald BN 20130125 00Z time+2 Forecast: Mslp_Precip(1 Hour); time+1 Forecast: Sfc Real. Humidity, 850hPa Wind, Geopotential Height Wind