Opened 2 years ago

Closed 2 years ago

#306 closed (fixed)

Update "ScpBufrToRaijin" to allow addition of Bufr files to tarballs

Reported by: Jin Lee Owned by: Jin Lee
Priority: major Component: ACCESS model
Keywords: APS3 Bufr remote copy sam MARS Cc: Susan Rennie, Chris Tingwell, Xudong Sun

Description

Currently if Bufr files for a new observation type are to be included in tarballs the tarballs need to be recreated. It would be convenient to simply add new Bufr files to existing tarballs. This ticket describes the work for adding this functionality to the package as well as some minor improvements.

Change History (6)

comment:2 Changed 2 years ago by Jin Lee

Owner: set to Chris Tingwell
Status: newassigned

Hi Chris,

Following changes were made to the package,

  • replaced MTSATCSR with AHICLEAR (Himawari-8 CSR)
  • added new functionality that allows concatenation of a Bufr tarball to a pre-existing archive on Raijin
  • more detailed commandline manpage

There is still an aspect of the package that needs to be improved but was not done in this ticket: when one or more Bufr tarballs are to be concatenated the ECMA archive from the relevant cycle is retrieved from sam regardless of whether the additional Bufr files come from MARS or not. Due to time constraint this defect is not tackled in this ticket. However this should be revisited later.

If you can review the code changes I would be grateful.

Cheers,

Jin

comment:3 Changed 2 years ago by Chris Tingwell

Hi Jin,

I've reviewed the revised script and tested the new functionality that allows concatenation of a Bufr tarball to a pre-existing archive on Raijin. The concatenation works properly.

However, there's still a problem with the specification of RemHostTwo: at the very least, it needs the nci.org.au domain added. Susan has also pointed out that it should be "r-dm", rather than "raijin".

One other suggestion: given that you've added variables to the input file, the short description of its contents in the command line help message is incomplete.

Cheers,
Chris.

Last edited 2 years ago by Chris Tingwell (previous) (diff)

comment:4 Changed 2 years ago by Chris Tingwell

Owner: changed from Chris Tingwell to Jin Lee

comment:5 in reply to:  3 Changed 2 years ago by Jin Lee

Replying to Chris Tingwell:

Hi Jin,

I've reviewed the revised script and tested the new functionality that allows concatenation of a Bufr tarball to a pre-existing archive on Raijin. The concatenation works properly.

However, there's still a problem with the specification of RemHostTwo: at the very least, it needs the nci.org.au domain added. Susan has also pointed out that it should be "r-dm", rather than "raijin".

  • Now the full DNS name used in RemHostTwo - sorry for forgetting this
  • I tried to use r-dm but the file transfer is extremely slow: 60 Kbps using r-dm vs. 60 Mbps using raijin login node. If you're happy I will revert to using raijin login node for file transfers.

One other suggestion: given that you've added variables to the input file, the short description of its contents in the command line help message is incomplete.

  • Updated commandline help

Cheers,
Chris.

comment:6 Changed 2 years ago by Jin Lee

Resolution: fixed
Status: assignedclosed

My development branch,

https://access-svn.nci.org.au/svn/nwp/branches/dev/jtl548/da/utilities/odb/scripts/r597_ScpBufrToRaijin_added_option@603

was merged into the "trunk",

https://access-svn.nci.org.au/svn/nwp/da/utilities/odb/scripts/ScpBufrToRaijin@604

The trunk version is installed in the following location on embery,

embery:/data/access/da/utilities/odb/scripts/ScpBufrToRaijin

Note: See TracTickets for help on using tickets.