wiki:access/JEDIAtNCI/MiniAcademyJune2020/Activity1B

Interactive activity 1B

This covers "Monday B: Running an Application, Git Flow, and ZenHub"

Steps 1

In the original tutorial all the scripts, configurations and data files were installed on AWS instances. Unfortunately these are not available. What we will do instead is to use some of the CTest unit tests which you built and ran in "Interactive activity 1A". To do this you will manually copy some files and then run example applications.

Here we will run an executable (fv3jedi_var.x) which was built in the previous exercise. You will also need a yaml configuration file (hyb-3dvar_gfs.yaml). A script to be qsub'ed (run_fv3jedi_var.bash) is provided. All the input files are under Data/ directory.

Prepare by running the following commands,

cd ${TOP_DIR}/academy/jedi/build      # go to the build directory
cd fv3-jedi                           # this is where fv3-jedi subproject was built
cd test                               # where CTest tests are
mkdir run-hyb-3dvar                   # where we will set up our tests
cd run-hyb-3dvar
cp ../../../bin/fv3jedi_var.x .       # copy one of the executables built
cp ../testinput/hyb-3dvar_gfs.yaml .  # copy a config file
cp -r --preserve=all ../Data .        # copy all the data files used in CTest unit tests - preserving timestamps
rm -fr Data/analysis/* Data/hofx/* Data/increment/*  # remove subdirectories to which fv3jedi_var.x writes output files - we don't want to be
                                                     # ..confused with CTest output with output from our own run
cp /home/548/jtl548/academy/public/scripts/shell/run_fv3jedi_var.bash .  # copy script that runs fv3jedi_var.x

Have a look at the yaml configuration file (explanation on yaml is in Lecture Tuesday, Feb 25 - System Configuration) and identify various input files including observations.

Instead of running ./run.bash hyb-3dvar as suggested in the Monterey tutorial use the script you copied to launch MPI processes,

qsub run_fv3jedi_var.bash

The outputs come out in the directory where you submitted the run_fv3jedi_var.bash script and the subdirectories underneath. Browse through some of the output files,

  • hyb-3dvar_gfs.run and hyb-3dvar_gfs.run.* - log output
  • Data/analysis/ - analysis files
  • Data/hofx/ - observations and model-equivalents at observation locations

Steps 2

In the Monterey academy participants used AWS instances where all the software packages were pre-installed. We don't have access to AWS so we will have to do this step differently.

All the Python scripts used in this step are available from the feature/academy branch of fv3-jedi. I have copied them so that participants can use them without checking them out. To use the Python scripts do,

module unload nwptools_py2                                          # unload Python2 modulefile
# Remove any other Python2 environment
module use /g/data3/hh5/public/modules                              # make sure you are a member of hh5 project
module load conda/analysis3                                         # for plotting we need cartopy
export PATH=/home/548/jtl548/academy/public/scripts/python:${PATH}  # add location of Python tools used in JEDI academy to PATH

Now use convergence.py to plot how cost function changes at each inner-loop iteration,

convergence.py --help                   # help on commandline options
convergence.py -l hyb-3dvar_gfs.run     # produce convergence plot

Use ImageMagick or your favourite graphics viewer to look at the plot. Try plotting other cost functions.

Next, produce some scatter plots of observation values and the background in observation space. E.g. for the brightness temperatures from channel 9 in AMSU-A/NOAA-19,

hofx.py -f Data/hofx/amsua_n19_hyb-3dvar-gfs_2018041500_m_NPROC.nc4 -v brightness_temperature_9@ObsValue

The model background,

hofx.py -f Data/hofx/amsua_n19_hyb-3dvar-gfs_2018041500_m_NPROC.nc4 -v brightness_temperature_9@hofx2

A plot of innovations,

innovation.py -f Data/hofx/amsua_n19_hyb-3dvar-gfs_2018041500_m_NPROC.nc4 -v brightness_temperature_9

Steps 4

We will need to produce an analysis increment from the full-field analysis produced from Step 1. To calculate analysis increment,

cp ../../../bin/fv3jedi_diffstates.x .  # copy the executable to run
cp ../testinput/diffstates_gfs.yaml .   # copy a config file
cp /home/548/jtl548/academy/public/scripts/shell/run_fv3jedi_diffstates.bash .  # copy script that runs fv3jedi_diffstates.x
qsub run_fv3jedi_diffstates.bash        # submit job script

After the job finishes check under Data/increment to check the file containing the analysis increment is produced,

ls -ltr Data/increment  # make sure the analysis increment is produced

Then plot the analysis increments of various fields. E.g.,

field_level_plot.py --input Data/increment/hyb-3dvar-gfs.20180415_000000z.nc4 --field T --field_lines DELP --field_barbs ua,va

Follow the rest of Step 4 of the Monterey tutorial.

Steps 5

As explained in the Monterey tutorial change your configuration file and see the effect on the analysis increment produced by the hybrid 3DVar.

To prepare for this exercise create a new directory where you will run the hybrid 3DVar using changed yaml configuration,

cd ${TOP_DIR}/academy/jedi/build/fv3-jedi/test  # change to the top-level fv3-jedi test directory
mkdir run-hyb-3dvar_step5                       # create a directory for this step
cd run-hyb-3dvar_step5

Repeat what you did in Step 1 by copying needed executable, yaml, data files, etc. Then follow the suggested changes described in Step 5 of the Monterey tutorial.

Steps 6

git-flow is not installed on Gadi. So use git commands to do this exercise.

Navigate to ${TOP_DIR}/academy/jedi/src/fv3-bundle/fv3-jedi and create a new feature branch. Give the feature branch a name that is meaningful and unique so that it doesn't conflict with anyone else's. I recommend that you use your name as a part of the branch name: e.g. feature/aircraft_temp_only_jinlee. At this stage you might like to read the relevant section of the main JEDI user document to review the git flow working practice.

For the code change make the same change you made to one of the yaml files in Step 5 but this time the change goes into the source. So, from ${TOP_DIR}/academy/jedi/src/fv3-bundle/fv3-jedi/test/testinput pick the yaml file you want to change. Make the same change that you did in Step 5. Commit the change and then push it to github.



To allow us to sign in to the jedi-ac06 github account from multiple, new devices JCSDA needs to approve it. I'm waiting on their approval. Until then you won't be able to do the following 2 steps!!!!

Steps 7

As suggested for this step in the Monterey tutorial open a web browser, log in to github, and navigate to the jedi-ac06/fv3-jedi repository.

Find your feature branch in the drop-down menu and issue a pull request.

Let's keep it simple. So for this mini academy we will not use ZenHub.

Review one another's work. When you have finished reviewing try merging pull requests - either your own or someone else's.

If you have not worked with github you may find the following github documentation helpful,

Steps 8

We will skip this exercise. However, if you are using your own Windows or Linux box and using Chrome or Firefox then try installing the ZenHub extension. You can then do this step as described in the Monterey tutorial. Have fun!

Last modified 4 weeks ago Last modified on Jun 10, 2020 11:55:23 AM