wiki:access/UMStandAloneExecs

Building the UM within Stand Alone Rose Suites

Rose has been designed not only to run the model but also build the model. Stand alone build suites are suites that contain all the FCM information required to build the UM and do not reference external make files as found in the access-svn repository. au-aa059 is a simple stand alone suite that can be used for building the UM model (v8.5). To run the suite, login onto accessdev and run the following commands.

rosie checkout au-aa059
rose suite-run -C ~/roses/au-aa059

This will be build the executable for UM 8.5. After running the suite, the default exectuable is located on raijin in;

$HOME/cylc-run/au-aa059/share/fcm_make/build-atmos/bin/um-atmos.exe

If you want to build the reconfiguration, there is a suite au-aa063 that builds the reconfiguration executable. To build the reconfiguration for vn8.5

rosie checkout au-aa063
rose suite-run -C ~/roses/au-aa063

This will be build the executables for UM 8.5. After running the suite, the default exectuables are located on raijin in;

$HOME/cylc-run/au-aa063/share/fcm_make/build-atmos/bin/um-recon.exe

How FCM works within Rose

On NCI machines the model is built in two steps. The first step fcm_make extracts the source code. The second step fcm_make2 builds the model on the compute nodes on raijin. It's possible to do the extraction on accessdev as an alternative location and then mirror it across to raijin if required.

FCM make files can read environment variables such as $ROSE_SUITE_NAME or $HOME in the following example. These can be to set alternative locations, compiler flags if required.

The make file is defined in a suite's FCM application. Eg, au-aa059/app/fcm_make/file/fcm-make.cfg

In the case of the au-aa059. The fcm make file contains the following lines.

################################################################################
#Compiliation Steps
step.class[preprocess-atmos] = preprocess
step.class[build-atmos]      = build

steps         = extract mirror
$MIRROR_TARGET{?} = $HOME/cylc-run/$ROSE_SUITE_NAME/share/fcm_make
mirror.target = $MIRROR_TARGET
mirror.prop{config-file.steps} = preprocess-atmos build-atmos

#Build Target
$atmos_exec{?}            = um-atmos.exe
build-atmos.target        = $atmos_exec 
build-atmos.target-rename = um_main.exe:$atmos_exec

################################################################################
#Extraction
extract.ns = um jules
extract.location[um]    = fcm:um-tr@vn8.5
extract.location[jules] = fcm:um-tr@vn8.5
extract.path-excl[jules] = /
extract.path-incl[jules] = includes/shared \
                           src/control/shared \
                           src/control/um \
                           src/initailisation/shared \
                           src/initialisation/um \
                           src/science

# Branches/working copies:
extract.location{diff}[um] = \
       fcm:um_dev/vn8.5/GlobalAtmos6p0@7678  \
       fcm:um_dev/vn8.5/local_changes@HEAD
extract.location{diff}[jules] = \
       branches/dev/mrd599/um8.5_GlobalLand6p0@387

################################################################################
# Pre Process definitions
$keys_atmos = A03_9C=a03_9c \
              A13_2A=a13_2a \
              A19_1B=a19_1b \
              C84_1A=c84_1a \
              C95_2A=c95_2a \
              C96_1C=c96_1c \
              C97_3A=c97_3a \
              C98_0A=c98_0a \
              UM_JULES=um_jules \
              L19_1A=l19_1a \
              C_LONG_LONG_INT=c_long_long_int \
              C_LOW_U=c_low_u \
              FRL8=frl8 \
              LINUX=linux \
              BUFRD_IO=bufrd_io \
              LITTLE_END=little_end \
              LFS=lfs \
              _LARGEFILE_SOURCE=_largefile_source \
              _FILE_OFFSET_BITS=64

preprocess-atmos.prop{fpp.defs} = $keys_atmos
preprocess-atmos.prop{cpp.defs} = $keys_atmos

################################################################################
# Special dependancies
build-atmos.prop{dep.o.special}[um/src/control/top_level/um_main.F90] = blkdata.o
build-atmos.prop{no-dep.f.module} = mpl \
                                    gcom_mod \
                                    netcdf \
                                    mod_prism_proto \
                                    mod_prism_put_proto \
                                    mod_prism_grids_writing \
                                    mod_prism_def_partition_proto \
                                    mod_prism_get_proto

################################################################################
# Compiler flags 
$fcflags_common = -g -nofpp -i8 -r8 -xHost -traceback
$fcflags_level  = -O2 -fp-model precise

$fcflags_omp{?} = 
$ldflags_omp{?} = 
$ldgcom_omp{?}  = 

$ldflags{?}     = -openmp -lgcom -lnetcdff -lnetcdf \
                \ -lgrib_api_f90 -lgrib_api $ldflags_omp

$flags_coupling{?}     = 
$fcflags_coupling{?}   = $flags_coupling
$fppflags_coupling{?}  = $flags_coupling
$ccflags_coupling{?}   = $flags_coupling
$cppflags_coupling{?}  = $flags_coupling
$ldflags_coupling{?}   = -lanaisg -lanaism -lpsmile.MPI1 -lfscint -lmpp_io -lscrip

$fcflags_overrides{?} =
$fcflags_all = $fcflags_common $fcflags_level $fcflags_omp $fcflags_overrides

preprocess-atmos.prop{cpp}       = cpp
preprocess-atmos.prop{cpp.flags} = 

build-atmos.prop{cc} = mpicc
build-atmos.prop{fc} = mpif90
build-atmos.prop{cc.flags}    = -g -O3 -std=gnu99
build-atmos.prop{fc.flags}    = $fcflags_all
build-atmos.prop{fc.flags-ld} = $ldflags


################################################################################
# Additional flags for coupled models:
$fc_cpl = $fcflags_coupling $fcflags_all
build-atmos.prop{fc.flags}[um/src/control/coupling/oasis3_atmos_init_mod.F90] = $fc_cpl
build-atmos.prop{fc.flags}[um/src/control/coupling/oasis3_grid.F90]           = $fc_cpl
build-atmos.prop{fc.flags}[um/src/control/coupling/oasis3_split_comm_mod.F90] = $fc_cpl
build-atmos.prop{fc.flags}[um/src/control/coupling/oasis3_geto2a.F90]         = $fc_cpl
build-atmos.prop{fc.flags}[um/src/control/coupling/oasis3_puta2o.F90]         = $fc_cpl
build-atmos.prop{fc.flags}[um/src/control/coupling/oasis3_get.F90]            = $fc_cpl
build-atmos.prop{fc.flags}[um/src/control/coupling/oasis3_put.F90]            = $fc_cpl

# Attempt to keep compile times down (excludes $fcflag_omp)
$fc_noomp = $fcflags_common $fcflags_level $fcflags_overrides
build-atmos.prop{fc.flags}[um/src/control/top_level/u_model.F90]     = $fc_noomp
build-atmos.prop{fc.flags}[um/src/control/top_level/initial.F90]     = $fc_noomp
build-atmos.prop{fc.flags}[um/src/control/top_level/atm_step.F90]    = $fc_noomp
build-atmos.prop{fc.flags}[um/src/control/top_level/u_model_4A.F90]  = $fc_noomp
build-atmos.prop{fc.flags}[um/src/control/top_level/initial_4A.F90]  = $fc_noomp
build-atmos.prop{fc.flags}[um/src/control/top_level/atm_step_4A.F90] = $fc_noomp
build-atmos.prop{fc.flags}[um/src/control/stash/st_diag3.F90]        = $fc_noomp

# Required to compile with OpenMP
build-atmos.prop{fc.flags}[um/src/atmosphere/dynamics_advection/idl_surface_setup.F90] =  \
           \ $fcflags_common $fcflags_level $fcflags_overrides

# Required to prevent OpenMP multi-thread crash
build-atmos.prop{fc.flags}[um/src/atmosphere/boundary_layer/excf_nl_9c.F90] = \
           \ $fcflags_common -O2 -fp-model strict $fcflags_omp $fcflags_overrides




Definition of variables

extract.location[um] => defines where the um source code and what version of the branch

extract.location[jules] => defines where the jules source code and what version of the branch

$keys_atmos => defines the ifdefs for the UM model.

steps => defines what steps do in the first fcm make step. "extract" extracts the code. "mirror" copies the source code the location specified in $MIRROR_TARGET.

$MIRROR_TARGET => Location of where the extracted source code should be sent to.

extract.location{diff}[um] => Apply any branches over the um base.

extract.location{diff}[jules] => Apply any branches over the jules base.

atmos_exec => Name of the UM executable file.

Log files

There are three types of log files that can be used for solving problems when compiling the UM.

Suite Logs

Suite log files contain top level information about what jobs failed and job submission. They are located on Accessdev in

$HOME/cylc-run/SUITENAME/suite/out
$HOME/cylc-run/SUITENAME/suite/log
$HOME/cylc-run/SUITENAME/suite/err

Job Logs

Job log files contain more specific infortion about running a task. They are located on both Accesdev and Raijin in

$HOME/cylc-run/SUITENAME/log/job/fcm-make.1.1.err
$HOME/cylc-run/SUITENAME/log/job/fcm-make.1.1.out

FCM Logs

FCM log file is a low level log that can information about about how specific files are compiled. They are located on Raijin

$HOME/cylc-run/SUITE/share/fcm_make/fcm-make.log

Last modified 6 years ago Last modified on Oct 10, 2014 2:28:35 PM