Central Ancillary Program
The Met Office Central Ancillary Program (CAP) creates external data files for the UM, e.g. land sea mask, orography, vegetation, soil. The CAP reads high resolution master data files, interpolates to the model grid and writes files in UM format.
We have a repository for the CAP at https://access-svn.nci.org.au/trac/cap/. Branches currently in use are 7.7 for generating New Dynamics files and 8.3 for generating ENDGAME files. There was also a significant change in the way the program is built, controlled and run between these versions.
CAP 8.3 documentation at https://accessdev.nci.org.au/umdocs/CAP_8.3/
CAP 7.7
https://access-svn.nci.org.au/trac/cap/browser/branches/dev/Share/VN7.7_local_changes
fcm co https://access-svn.nci.org.au/svn/cap/branches/dev/Share/VN7.7_local_changes ANCIL_VN7.7_local_changes
CAP requires the UM GCOM library but can work with most versions. Build configurations for vayu (still valid for raijin) and ngamai are in src/config/Ancil_Dev. The vayu one finds GCOM with a module while the ngamai version is hardcoded to use /g/sc/ophome/access/apps/gcom/GCOM3.5/ngamai/default_for_vn7.5/lib/.
The dev-ext.MACHINE.cfg files basically just specifies the build location ($WORKDIR/WORKING/linux here) while linux-ifort-MACHINE.cfg has the compiler options.
Building on raijin
module load intel-fc module load intel-cc module load openmpi/1.6.5 module use ~access/modules module load gcom/4.5 module load fcm fcm co https://access-svn.nci.org.au/svn/cap/branches/dev/Share/VN7.7_local_changes ANCIL_VN7.7_local_changes cd ANCIL_VN7.7_local_changes/src/config/Ancil_Dev fcm extract dev-ext.vayu.cfg cd $WORKDIR/WORKING/linux fcm build % ls bin ancil_aerosol ancil_orog ancil_smow ancil_vegfracsoil ancil_general ancil_ozone ancil_sstice central_ancillary.exe ancil_mask ancil_slt ancil_top fcm_env.ksh
Scripts in bin (ancil_*) have been modified to set paths etc appropriately for raijin and ngamai. All copied to ~access/apps/cap/7.7/bin/.
Building on ngamai
module load intel module load openmpi/1.6.5 module use ~access/modules module load fcm fcm co https://access-svn.nci.org.au/svn/cap/branches/dev/Share/VN7.7_local_changes ANCIL_VN7.7_local_changes cd ANCIL_VN7.7_local_changes/src/config/Ancil_Dev fcm extract dev-ext.ngamai.cfg cd $WORKDIR/WORKING/linux fcm build % ls bin ancil_aerosol ancil_orog ancil_smow ancil_vegfracsoil ancil_general ancil_ozone ancil_sstice central_ancillary.exe ancil_mask ancil_slt ancil_top fcm_env.ksh
Configuration files
The CAP comes with a set of grid and option namelists for all the Met Office domains, e.g. ~access/apps/cap/7.7/control/namelist/grids/
Master data files
raijin:~access/data/master
The master dataset includes the GLOBE orography and 1 degree and 30 minute resolution. The representation of Australia in this has many errors and should not be used (even for global models). See HighResolutionAustralianOrography.
Instead the scripts have been altered to add an option GLOBE30_AUS and the original data file made inaccessible.
Testing and validation
A simple test is to generate the N48 mask and orography using the Met Office options (note that this includes some explicit changes to the mask from the file ~access/apps/cap/7.7/control/changes/mask/mask_changes.n48). Repository directory src/scripts/create_files contains the Met Office jobs for various grids, including the options used, e.g. https://access-svn.nci.org.au/trac/cap/browser/branches/dev/Share/VN7.7_local_changes/src/scripts/create_files/n48/mask.ll has
export MASKOPTS="-m4 -k1 -l -3"
module load cap export DATASETS="mask" export MASKOPTS="-m4 -k1 -l -3" ancil_top -model n48 -queue express
This creates a batch job, $WORKDIR/ancil.vn7.7/n48/make_ancil_file and submits it. The script default job time and memory limits are suitable for creating the mask.
Compare with standard Met Office file distributed with the external release test job.
$UMDIR/vn8.5/normal/utils/cumf ~access/data/UM_Input/vn8.2/ancil/atmos/land_sea_mask/igbp/v1/qrparm.mask qrparm.mask
cumf_full output file should have
Field 1 : Stash Code 30 : LAND MASK (No halo) (LAND=TRUE) : Level 88 88 Field 1 has 0 LOGICAL Differences OK
Orography (pack 2 option gives 32 bit fields to match the Met Office version)
export DATASETS="orog" export OROGOPTS="-d GLOBE1" ancil_top -model n48 -queue express -memory 4 -pack 2
$UMDIR/vn8.5/normal/utils/cumf ~access/data/UM_Input/vn8.2/ancil/atmos/orography/globe1/v1/qrparm.orog qrparm.orog
This time the comparison is not quite exact.
Field 1 : Stash Code 33 : OROGRAPHY (/STRAT LOWER BC) : Level 8888 ITEM= 251 0.128055529785156E+04 (40940238A0000000) 0.128055541992188E+04 (40940238C0000000) ITEM= 468 -0.337153363227844E+01 (C00AF8E6A0000000) -0.337153387069702E+01 (C00AF8E6C0000000) NUMBER OF DIFFERENT VALUES = 2 MAXIMUM DIFFERENCE= 1.220703125000000E-004 AT PT. 251
CAP 8.3
This uses fcm make to build, so requires a more recent fcm (default module on raijin is ok). It uses some code from the UM and OPS repositories. In the Met Office set up it also builds GCOM but here we use our standard library.
Building on raijin
module load intel-fc module load intel-cc module load openmpi/1.6.5 module use ~access/modules module load gcom/4.5 module load fcm
To extract the appropriate branch directly from the repository
fcm make -f https://access-svn.nci.org.au/svn/cap/branches/dev/Share/ANCIL8.3_nci/fcm-make.cfg
Alternatively to use a locally modified copy of the code or of the build configuration files
fcm co https://access-svn.nci.org.au/svn/cap/branches/dev/Share/ANCIL8.3_nci cd ANCIL8.3_nci/src/config/Ancil_Dev
To use the local configuration files rather than the repository versions
export ANCIL_CFG_ARCH=$PWD/x86_64-ifort.cfg export ANCIL_CFG_TYPE=$PWD/opt.cfg export ANCIL_CFG_BUILD=$PWD/build.cfg export ANCIL_CFG_CORE=$PWD/core.cfg export ANCIL_CFG_DEP=$PWD/dep.cfg
To use the local extracted code (working copy) rather than the repository version
export ANCIL_REPOSITORY=$WORKDIR/ANCIL8.3_nci
Then run
fcm make
Either method extracts a local copy of the code in a subdirectory of the current directory called "extract" and mirrors it to $WORKDIR/ancil/build/Ancil_Dev/. To build
cd $WORKDIR/ancil/build/Ancil_Dev/ fcm make
This creates build/bin/central_ancillary.exe.
Running
The options for creating ancillary files are now in XML files. This is read by a wrapper script and used to set up the job that creates the file. See CAP_raijin
CAP 9.1
CAP is now built as part of a rose stem test.
