Opened 3 years ago

Closed 3 years ago

#274 closed (fixed)

Upgrade to cylc 6.10.0, rose 2015.05.0, fcm 2015.05.0

Reported by: Martin Dix Owned by: Martin Dix
Priority: minor Component: Accessdev Server
Keywords: Cc:

Description

Upgrade accessdev to latest releases and remove some unused older versions.

Change History (8)

comment:1 Changed 3 years ago by Martin Dix

Owner: set to Martin Dix
Status: newassigned

At the moment we have

cylc-5.4.14 rose/2014-05
cylc-6.3.0 rose/2015.02.0
cylc-6.4.1 rose/2015.04.1
cylc-6.7.1 rose/2015.10.1
cylc-6.7.2 rose/2015.11.0
cylc-6.9.1 rose/2016.03.0

Since the accessdev reboot all suites have used the default except for one suite of Jin’s that crashes with the latest versions but runs ok with cylc 6.4.1 (versions are logged to ~access/tmp/cylclog on raijin - see #192).

I propose we reduce the list to

cylc-6.4.1 rose/2015.04.1
cylc-6.9.1 rose/2016.03.0
cylc-6.10.0 rose/2016.05.0

Testing puppet configuration with mrd599/cylc-6-10.0.

Following user feedback, also keeping cylc-6.7.2, rose 2015.11.0.

Last edited 3 years ago by Martin Dix (previous) (diff)

comment:2 Changed 3 years ago by Martin Dix

The cylc configuration on accessdev sets (e.g. in /usr/local/cylc/cylc-6.9.1/conf/global.rc)

    [[raijin.*]]
        cylc executable = /projects/access/bin/cylc

and the rose configuration sets (e.g. /usr/local/rose/2016.03.0/etc/rose.conf)

[rose-home-at]
raijin.nci.org.au=/projects/access

At the moment raijin:~access/bin/cylc does module load cylc/wrapper and then tries to load the appropriate rose version based on the value of CYLC_VERSION and finally runs the cylc command.

On raijin ~access/apps/cylc/6.9.1/conf/job-init-env.sh is set to load the appropriate version of rose (#189)

module load cylc/wrapper
# Load matching rose module for this version of cylc
module load rose/2016.03.0
module load fcm/2016.02.0

so it's no longer necessary to do it in the script.

~access/apps/cylc/wrapper/bin/cylc defaults to using 5.4.14 if CYLC_VERSION isn't set because cylc 5 didn't set it. After 5.4.14 is removed it should default to ~access/apps/cylc/default (which should itself be updated).

rose suite-run on accessdev does

 ssh -oBatchMode=yes -oConnectTimeout=10 raijin bash --login -c \'ROSE_VERSION=2016.03.0\ /projects/access/bin/rose\ suite-run\ -v\ -v\ --name=au-aa398\ --run=run\ ...

On raijin, rose calls cylc get-global-config to get run directory paths.
Rose calls cylc directly, so that the cylc module should be loaded from the rose wrapper script. CYLC_VERSION isn't set at this point so it's just getting the default version. Since it's only getting the path from the configuration this doesn't matter.

After we get rid of rose 2014-05, cylc 5.4.14, it's sufficient for ~access/bin/cylc to have

module use ~access/modules
module load cylc/wrapper

cylc $*

and ~access/bin/rose

if [[  -f /projects/access/modules/rose/${ROSE_VERSION} ]]; then
  module load rose/$ROSE_VERSION
else
  >&2 echo "Error: No module for rose/$ROSE_VERSION found on raijin"
  exit 1
fi

module load cylc/wrapper

rose $*

This was tested with modified paths on accessdev-test.

Note that it's not necessary to have ~access/bin in the raijin path.

Last edited 3 years ago by Martin Dix (previous) (diff)

comment:3 Changed 3 years ago by Martin Dix

After updating accessdev noticed very high CPU usage from cylc GUI processes, reaching 40-90%.

On accessdev-test, the GUI for the trivial au-aa398 uses about 8% compared to a negligible amount with cylc 6.9.1.

Temporarily set the defaults back to cylc 6.9.1 and rose 2016.03.0.

See https://github.com/cylc/cylc/issues/1840

Last edited 3 years ago by Martin Dix (previous) (diff)

comment:4 Changed 3 years ago by Martin Dix

Susan also noticed that job.out no longer had the PBS usage at the end. Changing the default version of cylc loaded by the wrapper when CYLC_VERSION isn't set back to 5.4.14 fixed this, even though this isn't the version that the task actually runs with!

comment:5 Changed 3 years ago by Martin Dix

Cylc GUI problem fixed with 6.10.1.

Problem with the PBS usage fixed by defaulting to a specific version in ~access/apps/cylc/wrapper/bin/cylc rather than the link default. E.g.

    if [[ -n ${CYLC_VERSION:-} && -d $CYLC_HOME_ROOT/$CYLC_VERSION ]]; then
        CYLC_HOME=$CYLC_HOME_ROOT/$CYLC_VERSION
    # No default - CYLC_VERSION should always be set from accessdev
    else
         CYLC_HOME=$CYLC_HOME_ROOT/6.10.1
    fi

Still not clear why the default link doesn't work.

comment:6 Changed 3 years ago by Martin Dix

All cylc commands (on both accessdev and raijin) give warnings like

/usr/lib64/python2.6/site-packages/jinja2/__init__.py:31: UserWarning: Module Pyro was already imported from /usr/local/cylc/cylc-6.10.1/lib/Pyro/__init__.py, but /usr/lib/python2.6/site-packages is being added to sys.path
  __version__ = __import__('pkg_resources') \

This is because cylc 6.10 uses its own modified version of pyro for better performance (https://github.com/cylc/cylc/pull/1755).

We can't get rid of the version in /usr/lib/python2.6/site-packages/ while we have earlier versions of cylc.

Last edited 3 years ago by Martin Dix (previous) (diff)

comment:7 Changed 3 years ago by Martin Dix

Rose 2016.05.1 has been released but doesn't have anything that seems worth upgrading from 2016.05.0 immediately.

comment:8 Changed 3 years ago by Martin Dix

Resolution: fixed
Status: assignedclosed
Note: See TracTickets for help on using tickets.