wiki:access/opdata_on_nci

Operational ACCESS-G/R/C/TC data on raijin

Operational ACCESS-G/R/C/TC data in

ngamai:/g/sc/opdata/access_data/access_X/samvX/archive/samvX (X=g/r/tc)
ngamai:/g/sc/opdata/access_data/access_c/CYY11/running/um-output/datam (YY=AD/BN/DN/PH/SY/VT)
ngamai:/g/sc/opdata/access_postproc/pp_ACCESS-X_expver_yyyymmddhh (X=G/R/TC/AD/BN/DN/PH/SY/VT)

and SAM are now regularly copied over to raijin as follows

1. model output pX files (X=a..g)

are being converted to grib and stored on raijin at

raijin:/g/data/rr4/opendap/access-X/1/yyymmdd/hhmn (X=g/r/tc/ad/bn/dn/ph/sy/vt)

under 4v/ an/ fc/ structure to facilitate MARS/OpenDAP interface.

These grib files can be converted to NetCDF format by "g2nc", or retrieved in a mars request, with

export GRB_ROOT=/g/data/rr4/opendap

and normal MARS retrieval requests.

2. pressure level fields as calculated by BNOC

are copied directly from /g/sc/opdata/access_postproc in grib format and stored at

raijin:/g/data/rr4/opendap/access-X/1/yyymmdd/hhmn (X=g/r/tc/ad/bn/dn/ph/sy/vt)

as pcNNN.grb under an/ and fc/

Retrieval via mars as per above (with levtype = pl)

3. archived data on sam

are first recovered and then rsync'ed to raijin at

raijin:/g/data/rr4/samnmc/access-X/yyyy/mm

in the same directory structure as on sam, ie.

sam:/samnmc_glob/access-g/yyyy/mm

an_smc.yyyymmddhh00
analysis-increments_yyyymmddhh00
bgerr.yyyymmddhh
bk.ppsmc.yyyymmddhh00
bufryyyymmddhh.tar.gz
cx.yyyymmddhh.tar.gz
cx.background.yyyymmddhh00
ECMAyyyymmddhh.tar.gz
environment.yyyymmddhh.tar.gz
frame_access_r.yyyymmddhh.tar.gz
iau-start-dump.yyyymmddhh00
iau0-dump.yyyymmddhh00
satradbiases.yyyymmddhh.tar.gz
screen-var.yyyymmddhh.tar.gz
varobs.yyyymmddhh.tar.gz
web.yyyymmddyy.tar.gz

ls_fields_yyyymmddhh/

qwxbgza_ca0NN.yyyymmddhh (nn=06..12)

pi_yyyymmddhh/

qwxbgza_piNNN_yyyymmddhh_utc_fc.um

sam:/samnmc_reg/access-r/yyyy/mm

an_smc.auslam.yyyymmddhh
analysis-increments_yyyymmddhh00
bufryyyymmddhh.tar.gz
cx.yyyymmddhh.tar.gz
ECMAyyyymmddhh.tar.gz
environment.yyyymmddhh.tar.gz
frame_seqld.yyyymmddhh.tar.gz
frame_sydney.yyyymmddhh.tar.gz
frame_victas.yyyymmddhh.tar.gz
frame_adelaide.yyyymmddhh.tar.gz
frame_darwin.yyyymmddhh.tar.gz
frame_perth.yyyymmddhh.tar.gz
iau0-dump.yyyymmddhh00
satradbiases.yyyymmddhh.tar.gz
varobs.yyyymmddhh.tar.gz
web.yyyymmddhh.tar.gz

input_yyyymmddhh/

an_smc.yyyymmddhh
lbc_yyyymmddhh_alabc_aus
qwqg12.yyyymmddhh.bgerr

pi_yyyymmddhh/

qwxbeta_piNNN_yyyymmddhh_utc_fc.um

sam:/samnmc/access_c/YY/yyyy/mm (YY=adelaide/darwin/perth/seqld/sydney/victas) astart.yyyymmddhh
lbc_yyyymmddhh_alabc_aus
um_environment.yyyymmddhh.tar.gz

3. data not yet copied over to raijin

as they are not archived in sam:

obs-monitoring/
ops-listings/

access-g

analysis-fields/ ice.glb.recon, sst.glb.recon

access-r

analysis-fields/ analysis-pf
cycle_dumps/ls-fields-dir/ qwxbeta_ca0NN


cron jobs

are running roughly an hour after the model finished,

03 01 * * * /g/sc/home/ttl/cron/nci_data.sh access-g 18 2>&1 > /dev/null
40 03 * * * /g/sc/home/ttl/cron/nci_c.sh 00 2>&1 > /dev/null
00 04 * * * /g/sc/home/ttl/cron/nci_data.sh access-r 00 2>&1 > /dev/null
03 07 * * * /g/sc/home/ttl/cron/nci_data.sh access-g 00 2>&1 > /dev/null
40 09 * * * /g/sc/home/ttl/cron/nci_c.sh 06 2>&1 > /dev/null
00 10 * * * /g/sc/home/ttl/cron/nci_data.sh access-r 06 2>&1 > /dev/null
03 13 * * * /g/sc/home/ttl/cron/nci_data.sh access-g 06 2>&1 > /dev/null
40 15 * * * /g/sc/home/ttl/cron/nci_c.sh 12 2>&1 > /dev/null
00 16 * * * /g/sc/home/ttl/cron/nci_data.sh access-r 12 2>&1 > /dev/null
03 19 * * * /g/sc/home/ttl/cron/nci_data.sh access-g 12 2>&1 > /dev/null
40 21 * * * /g/sc/home/ttl/cron/nci_c.sh 18 2>&1 > /dev/null
00 22 * * * /g/sc/home/ttl/cron/nci_data.sh access-r 18 2>&1 > /dev/null

where nci_data.sh is the driving script to loop over the model output pX (X=a..g) files in parallel, it calls the main script nci_send_data.sh to convert/send ACCESS-G/R data to raijin


Bureau archives of ACCESS-G2 start files

More information regarding the location of ACCESS-G2 start-files archived on Bureau of Meteorology filesystems is available here: G2_data_at_BoM.

Last modified 3 years ago Last modified on Oct 5, 2016 3:23:26 PM