Changes between Version 18 and Version 19 of access/BomAccessDocumentation/bom-nwptools


Ignore:
Timestamp:
Jun 24, 2019 2:52:33 PM (4 weeks ago)
Author:
Vinod Kumar
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • access/BomAccessDocumentation/bom-nwptools

    v18 v19  
    8484== Name of the environment ==
    8585
    86 We propose "nwppytools" as the package name. There is a possibility that we could eventually end up with a total of 4 versions. There is a requirement to support both python2 and python3. Also, as a release strategy, it would be easier to manage if there is an unstable version for each of the above packages. The module could be potentially named as:
     86We propose "nwp-pytools" as the package name. There is a possibility that we could eventually end up with a total of 4 versions. There is a requirement to support both python2 and python3. Also, as a release strategy, it would be easier to manage if there is an unstable version for each of the above packages. The module could be potentially named as:
    8787
    8888{{{
    89 conda/nwppytools27
    90 conda/nwppytools27-unstable
    91 conda/nwppytools3
    92 conda/nwppytools3-unstable
     89conda/nwp-pytools27
     90conda/nwp-pytools27-unstable
     91conda/nwp-pytools3
     92conda/nwp-pytools3-unstable
    9393}}}
    9494
    95 Note: It would be worth having a tag for a particular nwppytools version, indicating which version of the Met Office "scitools" version it corresponds to. For example, the initial version of "nwppytools" seems to be working for suites which use "scitools/production-os-41-1" at the Met Office. Hence, a suggested name would be nwppytools27-production-os41-1 or nwppytools27-os41-1 (if collective preference is for a shorter name).
     95Note: It would be worth having a tag for a particular nwp-pytools version, indicating which version of the Met Office "scitools" version it corresponds to. For example, the initial version of "nwp-pytools" seems to be working for suites which use "scitools/production-os-41-1" at the Met Office. Hence, a suggested name would be nwp-pytools27-production-os41-1 or nwp-pytools27-os41-1 (if collective preference is for a shorter name).
    9696 
    9797
    9898== Python 2.x.x environment ==
    9999
    100 The following is an initial list of packages for conda/bom-nwptools27 environment, given in the format of yml file. There are obviously a few packages that are not in the channels. Claire recommend we create our own channel. Otherwise, this has to be installed out of the source code directly.
     100The following is an initial list of packages for conda/nwp-pytools27 environment, given in the format of yml file. There are obviously a few packages that are not in the channels. Claire Carouge from the CoE recommend we create our own channel. Otherwise, this has to be installed out of the source code directly.
    101101
    102102There are a few things to be noted:
    103    1. The package called mule in the anaconda cloud is quite old (vn 1.2) and the um_utils doe ot exits. So these packages were installed manually from the source files downloaded off the GitHub.The source files I downloaded (vn 2017.08) can be found under:
    104 
     103   1. The mule package in the anaconda cloud is quite old (vn 1.2) and the um_utils does not exists. So these packages were installed manually from the source files downloaded off the GitHub.The source files I downloaded (vn 2017.08) can be found under:
     104
     105    {{{
    105106    Raijin:/g/data/dp9/vvk548/softwares/sources
    106 
    107     Extract them to your personal space and install them using the setup.py file, using your activated anaconda environment. I.e., go to the directory where the sources are unzipped, and type:
    108 
     107    }}}
     108
     109    To install them into a conda environment, activate the conda environment, extract the gzip file somewhere in your personal space and then install them using the setup.py fil. i.e., go to the directory where the sources are unzipped, and type:
     110
     111    {{{
    109112    module load conda
    110113    conda activate <conda env name>
    111114    conda remove mule (if you have them installed already)
    112115    python setup.py install
    113 
    114     Do the same for um_utils as well. You can verify the install by "conda list".
    115 
    116   2. The iris version that works with both GES and RES is found to be iris=1.13.0. Any version later than this throw errors in relation to the dask module. Basically, lateversion of iris calls dask which inturn calls numpy.cumprod function, available only on numpy 1.5.x or later. However, upgrading to numpy=1.6.4 throw some obscure error and hence we had to revert to the earlier numpy=1.11.3 version. However, the iris=1.13.0 is not compatible with eccodes and cdo. Hence we had to omit these two modules from our installation. 
     116    }}}
     117
     118    You can verify the install by "conda list".
     119
     120  2. The iris version that works with both GES and RES is found to be iris=1.13.0. Later versions of iris (tested 2.0.0) were found to have a compatibility issue with numpy by calling the dask module. Basically, later version of iris calls dask which inturn calls {{{numpy.nancumprod}}} function, available only on numpy 1.13.0 or later. However, upgrading to a later version (numpy1.16.4) resulted in an unstable installation with further dependency issues. Hence, we reverted back to the earlier numpy=1.11.3 version. However, the iris=1.13.0 is not compatible with eccodes and cdo packages available via the conda-forge channel. Hence we had to omit these two modules from our installation. 
    117121
    118122{{{
    119 name: bom-nwptools27
     123name: nwp-pytools27
    120124 - _r-mutex=1.0.0=anacondar_1
    121125  - affine=2.2.2=py27_0