Automated Rose Stem Testing with Jenkins

ARCCSS CMS has set up a Jenkins server on the NCI cloud to automatically test changes made to the trunk of the UM (and other models)

You can find the Jenkins server at Note you will need to log in using your NCI credentials in order to see the UM test results.

What is Jenkins?

Jenkins is an open source continuous integration (CI) tool, used to automatically build a program, run its tests, then report on the test results. We use Jenkins to automatically run the UM's Rose Stem test suite every night, to make sure changes contributed to the model trunk aren't causing the model to break on Raijin. If tests fail then the ACCESS infrastructure admins are notified, they can then identify if this is due to a change breaking or an expected alteration to the model output.

What tests are run?

Jenkins runs the nightly test group on trunk every night, and the weekly test group every Saturday. At the time of writing the nightly test group includes two N48 endgame runs, one with the IO server enabled and one without, as well as the developer tests. The weekly test group includes all tests currently available at NCI. More details on NCI test apps are available at

How does it work?

To run the tests the Jenkins server connects to Accessdev as the 'accesstester' user. It then checks out the latest UM trunk and runs rose stem. Once the tests have finished it runs a script to collect the results from all tasks, then copies the test results and the trac.log file back to the server. If any tests have failed Jenkins then sends out emails to the admins. Current list is Scott Wales, Martin Dix, Robin Bowen, Wenming Lu.

Adding new tests

To add new tests to the Jenkins run contribute them to the UM trunk, making sure they're included in the nightly or weekly groups. Members of the 'access.admin' NCI group are also able to get Jenkins to run tests on a specific branch, by going to the UM/branch job and starting a new build.

Last modified 4 years ago Last modified on Feb 22, 2019 5:58:17 PM