Opened 15 months ago

Last modified 15 months ago

#348 new

Log file archiving

Reported by: Martin Dix Owned by:
Priority: major Component: ACCESS-CM2
Keywords: Cc:

Description

Coupled model log files can be very large.

Rose has the ability to automatically tar and gzip.

Change History (2)

comment:1 Changed 15 months ago by Martin Dix

In Dave's u-ar527, log file from a 6 month run is 67 MB!

% ls -l 00200101T0000Z/coupled/01     
total 68388
-rwxr-xr-x 1 dhb599 p66     8210 Nov  7 01:58 job
-rw-r----- 1 dhb599 p66   889959 Nov  7 05:19 job.err
-rw-r----- 1 dhb599 p66 69114325 Nov  7 05:22 job.out
-rw-r--r-- 1 dhb599 p66      237 Nov  7 05:22 job.status

Splitting the job.out file into parts from the separate models shows that only trivial amounts are from cylc and CICE and that there's about 50 MB from MOM and 19 from the UM.

comment:2 Changed 15 months ago by Martin Dix

Using the archive-logs-at option in rose-prune creates a .tar.gz file on the server (accessdev) and deletes the log files on raijin.

For this to be useful we'd need to greatly increase the size limit on log files that are copied back to accessdev.

Note: See TracTickets for help on using tickets.