Changes between Initial Version and Version 1 of access/n512_nci


Ignore:
Timestamp:
Oct 10, 2014 2:29:28 PM (5 years ago)
Author:
trac
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • access/n512_nci

    v1 v1  
     1----
     2{{{
     3#!html
     4<h1 style="text-align: center; color: blue"> APS2-development ACCESS-G N512 25km resolution global NWP medium-range assimilation-forecast suite documentation</h1>
     5<h1 style="text-align: center; color: red"> WORK IN PROGRESS</h1>
     6}}}
     7----
     8
     9=== External Links ===
     10
     11 * [http://www.cawcr.gov.au/research/esm/index.php CAWCR Earth System Modelling]
     12
     13== 1. Brief Description ==
     14
     15  ACCESS Global N512 suite is developed by ACCESSS Model System Team (MST), in collaboration with other ACCESS teams. Comparer with previous ACCESS-G suite, it has an increased horizontal resolution of 25 km (from 40km), and the same 70 level of vertical resolution (to 70 km?). The System were originally developed on vayu at NCI in August 2012, and it was ported to Solar at BOM in December 2012. The systems were again ported to the new HPC systems both at NCI and BOM in 2013, and they are now routinely running on raijin at NCI and on Ngamai at BOM. Some of the recent verifications are available at [http://flurry-bm/~yix/aps2_n512_plots], and early ones are at [http://flurry-bm/~yix/n512_plots]. Charts from N512 suite can be viewed at (put link here)
     16 
     17== 2. Suite Specification ==
     18
     19 
     20  The Configuration of ACEESSS N512 suite is based on PS32 release from the UK Met Office. It runs 4 times a day. At each cycle (00Z,06Z,12Z,18Z), observations are processed by '''OPS''' (observation is handled via '''ODB'''), followed by a '''4DVAR''' Analysis. The soil moisture content (smc) is created by ''' SURF ''', and updated at each cycle. In addition, the sea surface temperature (sst, from !BlueLink) and the sea ice (from NCEP) are updated at 06Z daily. At 00Z and 12Z, a 10 days forecast is produced by the Unified Model (UM).
     21
     22 Specific details about the N512 suite are summarised in  Table below.
     23 || ''' Components ''' || ''' Horizontal Grid ''' || ''' Vertical Grid  ''' || ''' Time Step ''' || ''' Software Versions '''|| ''' Comments '''||
     24 || ''' UM '''         || 1024 x 768 ( ~25km)             ||    L70                 ||  10 minutes        ||      8.2                 ||               ||
     25 || ''' VAR '''        || 432  x 325              ||    L70                 || 20(?) minutes        ||      29.1.2              || Using V30.0 is being tested ||
     26 || ''' OPS '''        ||                         ||    L70                 ||                   ||      30.0.0              || !ObsGroups :  Surface, Aicraft, AMV, Scat, ATOVS,AIRS,MTSATClear,CRIS,ATMS,SSMIS ||
     27 || ''' SURF '''       ||                         ||                        ||                   ||      18.5                || Testing SURF30.0 ||
     28 || ''' ODB  '''       ||                         ||                         ||                   ||     ODB30               || With ODB feedBack Enabled ||
     29
     30Note that, the N512 suite has gone through a few updates since it started in 2012. Version of various software used may still be changed.
     31
     32
     33
     34== 3. Suite Dependencies ==
     35
     36As a complex NWP system, the N512 suite contains many tasks. Some of the tasks are associated with components mentioned in previous section, others are just simple house keeping tasks, which gets inpuns in places or sends outputs away. A scheduler controls when and how to run a task within the suite. Depending on the scheduler being used, there are two types of ACCESS N512 NWP suites, one is under '''SCS''' (Suite Control System from UKMO), the other is under '''cylc''' (from NIWA). The SCS suite is only available at the Bureau, the cylc suite is available both at NCI and atthe Bureau. Although the
     37two types of N512 suites share most aspects, there are some differences: [[BR]]
     38 * The SCS suite is hosted on a HPC (''Ngamai'');
     39   The cylc suite is hosted on a none-HPC machine: ''accessdev'' at NCI, and ''twister'' at the Bureau.[[BR]]
     40 * Component tasks (''see the full task list in '''3.1''' below for more details '') '''OPS/VAR/UM/SURF ''' maybe defined differently [[BR]]
     41   * For SCS suite, '''OPS/VAR/UM''' tasks are defined using the corresponding UI (from '''UKMO'''); [[BR]]
     42     ''' SURF ''' tasks are defined within '''SCSUI''' itself. [[BR]]
     43   * For cylc suite, '''OPS/VAR/SURF''' tasks are defined with ''rose'' (from '''UKMO''') through ''rose-app.conf'' files. [[BR]]
     44     ''' UM ''' tasks are still defined using '''UMUI''' in current APS2 N512 suite. This is due to the old version of UM (vn8.2) which does not have ''rose'' support. [[BR]]
     45 * Another noticable difference between the '''SCS''' and '''cylc''' suite is that [[BR]]
     46   * '''SCS''' suite runs one cycle at a time, such 2014040106;[[BR]]
     47   * '''cylc''' suite can run across multiple times at once, ie tasks have their own cycle times. This means that ''' cylc ''' suite is far more efficient, especailly in research environment with none real time running or in catch up mode in operation.
     48 
     49=== 3.1 Suite Tasks ===
     50 There are many tasks within the ACCESS N512 suite. Roles of these tasks are listed as below [[BR]]
     51{{{
     52 |-on_local
     53   |  |-Prep                 send bin and app over (cylc suite only)
     54 |-on_remote   
     55 | |-remote_cycling       
     56 |   |-get_bufr           get obs bufr files
     57 |   |-get_old_bgerr      get old bgerr file ( only at cold start )
     58 |   |-get_ic             get external start dump file ( only at cold start )
     59 |   |-get_surf           get external sst and sea ice files (at 06Z only)
     60 |   |-ODB                ODB family
     61 |   | |-get_iasi         Create ODB for iasi
     62 |   | |-get_airs         Create ODB for airs
     63 |   | |-get_surface      Create ODB for surface
     64 |   | |-get_cris         Create ODB for CrIS
     65 |   | |-get_s            Create ODB for Screen OPS
     66 |   | |-get_atms         Create ODB for ATMS
     67 |   | |-get_scat         Create ODB for Acat
     68 |   | |-get_mtsatclear   Create ODB for MTSatClear
     69 |   | |-get_gpsro        Create ODB for GPSRO
     70 |   | |-get_atovs        Create ODB for ATOVS
     71 |   | |-get_as           Create ODB for Aircraft and Sonde
     72 |   | |-get_amv          Create ODB for SatWind
     73 |   | |-get_ssmis        Create ODB for SSMIS
     74 |   |-build_ops          Build ops executable
     75 |   |-GEN                GEN Family
     76 |     |-bgerr            Create backgrund Error
     77 |     |-OPS              OPS family
     78 |       |-Ops_iasi       Extract and process iasi
     79 |       |-Ops_amv        Extract and process SatWind
     80 |       |-Ops_airs       Extract and process airs
     81 |       |-Ops_surface    Extract and process Surface
     82 |       |-Ops_cris       Extract and process CrIS
     83 |       |-Ops_s          Extract and process Screen Obs
     84 |       |-Ops_atms       Extract and process ATMS
     85 |       |-Ops_mtsatclear Extract and process MTSATClear
     86 |       |-Ops_gpsro      Extract and process GPSRO
     87 |       |-Ops_atovs      Extract and process ATOVS
     88 |       |-Ops_as         Extract and process Aircrafe and Sonde
     89 |       |-Ops_scat       Extract and process ASCAT
     90 |       |-Ops_ssmis      Extract and process SSMIS
     91 |     |-VAR              VAR family
     92 |     | |-ConfigScreen   Configure Screen LS files
     93 |     | |-ConfigLS108    Configure LS for N108 files
     94 |     | |-ConfigLS216    Configure LS for N216 files
     95 |     | |-ScreenVAR      3D Screen Analysis
     96 |     | |-Analysis216    N216 4D AnalysisPF
     97 |     | |-Analysis108    N108 4D AnalysisPF
     98 |   |-SURF               SURF family
     99 |   | |-Surf_ssticesnow  produce sst/seaice/snow at targeted grid (at 06Z only)
     100 |   | |-Surf_smc         produce smc at targeted grid
     101 |   |-UM                 UM family
     102 |   | |-g2g              reconfigure start dump to targeted grids ( only at cold start )
     103 |   | |-FC_short         short step IAU Forecast
     104 |   | |-FC               IAU Forecast
     105 |   | |-InitialFC        Initial Forecast ( only at cold start )
     106 |   | |-ReconSST         update sst and seaice (at 06Z only)
     107 |   |-Archive            Archive family
     108 |   | |-run_archive      run time archive family
     109 |   |   |-a_pa           run time archive pa stream to mars/opendap
     110 |   |   |-a_pb           run time archive pb stream to mars/opendap
     111 |   |   |-a_pc           run time archive pc stream to mars/opendap
     112 |   |   |-arc_staging    run time staging
     113 |   | |-post_fc          post UM forecast family
     114 |   | | |-PostFinal      last time archive
     115 |   | | |-staging        move files to archive directory
     116 |   | | |-charts         produce forecast charts
     117 |   | | |-PostArchive    archive to sam/mdss  for previous cycle
     118 |   | | |-housekeeping   tidy up
     119}}}
     120
     121Dependencies among tasks are illustrated as below, where nodes '''ODB''' and '''OPS''' are ''family'' nodes, they represent tasks within the family detailed as above.
     122
     123
     124                                                       [[Image(n512.png, 50%)]]
     125
     126The suite dependency graph also shows the data flow within the suite, which is useful for a user to understand the suite, especial when a manual action (outside the scheduler) is required, such as ''trigger'' a task, ''reset a task state'' in a '''cylc''' suite.
     127
     128=== 3.2 Component Definitions ===
     129
     130As mentioned early in Section 3, tasks from components '''OPS/VAR/SURF/UM ''' are defined either through ''UIs '' (User Interface), or with rose task configuration ''rose-app.conf'' . When they are defined with '''rose''', definitions of these component tasks are a  part of the '''cylc''' suite definition.
     131When they are defined though ''UIs'', they are referenced by the '''SCS''' suite, but are not included in the '''SCSUI''' suite definition.
     132
     133Component definitions that we talked about here are only applied to tasks defined through ''UIs''. A corresponding ''rose-app.conf'' file can be obtained directly from a ''UI'' definition. Please note that, beyond APS2, ''UIs'' will be replaced by ''rose'', and will no longer be supported.
     134
     135 * '''OPS tasks '''
     136   * ''' under svn :''' [svn://ngamai04/access/svnrep/nwp/trunk/OPS/components/Ops_OperComps_gl/v3][[BR]]
     137     Note that a svn location on raijin is to be defined
     138   * ''' on disk   :''' ''~access/nwpdir/share/APS2/OPS/components/Ops_OperComps_gl/v3 ''
     139 * '''VAR tasks '''
     140   * ''' under svn : ''' [svn://ngamai04/access/svnrep/nwp/trunk/VAR/components/Var_OperComps_gl/v2] [[BR]]
     141   * ''' on disk   : ''' '' ~access/nwpdir/share/APS2/VAR/components/Var_OperComps_gl/v2 ''
     142
     143 * '''UM tasks '''
     144   * ''' under svn : ''' [svn://ngamai04/access/svnrep/nwp/trunk/um/components/N512/v3] [[BR]]
     145   * ''' on disk   : ''' '' ~access/nwpdir/share/APS2/um/components/N512/v3 ''
     146
     147Note that '''SURF ''' tasks are part of the suite definition, both under ''' SCS ''' and ''' cylc ''', and they are kept with the suite database under '''SCSUI''' (''on Ngamai'') or '''rosie''' (''on accessdev'').
     148
     149=== 3.3 Suite Control Data ===
     150
     151Within a NWP suite, tasks usually require some control data in order to be executed, such as '''!StationList''' for OPS, '''Covariance File''' for VAR, '''Ancillary Files ''' for UM etc.
     152These control data are under '''~access''', and small ones are also under '''svn'''. Details are as below. [[BR]] 
     153  * ''' OPS and VAR'''
     154    * ''' Small files  under svn : ''' [[BR]]
     155    [svn://ngamai04/nwp/browser/trunk/OPS#control] [[BR]]
     156    [svn://ngamai04/nwp/browser/trunk/VAR#control] [[BR]]
     157    * ''' On Disk : ''' [[BR]]
     158                      '' ~access/nwpdir/share/APS2/OPS/control '' [[BR]]
     159                      '' ~access/nwpdir/share/APS2/VAR/control '' [[BR]]
     160                      '' ~access/nwpdir/share/APS2/Data ''  : large data files for ''' OPS '''[[BR]]
     161                      '' ~access/nwpdir/share/APS2/other '' : large data files for ''' VAR ''' [[BR]]
     162    Note that files that are under ''~access/nwpdir/share/APS2/Data'' and ''~access/nwpdir/share/APS2/other '' are large files, and they are not maintained ''svn''
     163 * ''' UM '''
     164   '''UM''' control files stay on disk.
     165       * version specific control files are under [[BR]]
     166          ''~access/umdir/vn$UM_VERSION ''
     167       * Resolution dependent files are under ( to be decided )   [[BR]]
     168          ''~access/umdir/ancil''
     169=== 3.4 Component Software ===
     170
     171== 4. Running The Suite ==
     172== 5. Data Management ==