Assess possible degradation of TC forecasts by ACCESS-G caused by not assimilating some observation types

Bureau currently does not receive and so its models do not assimilate a large number of observation types that UKMO doesn. This is thought to degrade Bureau's global model forecasts.

There is a view that the observation coverage is sufficient (Kelly et al. 2007).

However Fiorino (2009) showed that changes (in this case IFS physics) may not show up in gross verification statistics but can show up in tropics: e.g. tropical winds, tropical cyclone tracks.

Also the impact of one observation type on its own may not be clear. However the synergistic effect of a number of observation types can be magnified.

Additional observation types

Following two tables show a comparison of observations which are currently used operationally in ACCESS-G3 and those used in UKMO OS42,

In-situ observations

Obstype	Usage in ACCESS-G3	Usage in UKMO OS42	Comment
METAR, SYNOP	used	used
TEMP, PILOT, SONDE	used	used
AIREPS, AMDARS	used	used

Satellite observations

Obstype	Usage in ACCESS-G3	Usage in UKMO OS39	Comment
ATOVS	MetOp-1, MetOp-2, NOAA-15, NOAA-18, NOAA-19	MetOp-1, MetOp-2, NOAA-15, NOAA-18, NOAA-19
AIRS	Aqua	Aqua
IASI	MetOp-1, MetOp-2	MetOp-1, MetOp-2
CrIS	S-NPP	S-NPP
ATMS	S-NPP	S-NPP
AMSR	GCOM-W1	GCOM-W1
SSMIS	DMSP17, DMSP18	DMSP17, DMSP18
GMI	not used	GPM	UKMO OS40 started assimilating GMI from 20180213T06
MWTS-1, MWHS-1	not used	FY-3B, FY-3C	In OPS app config MetDB subtype is ATOVS for FY-3B and MWTS for FY-3C; Q. Does this mean MWHS is not used?
Satwind	METEOSAT-8, METEOSAT-11, H-8, GOES-15, GOES-16, Aqua	METEOSAT-8, H-8, GOES-15, MetOp-1, MetOp-2, NOAA-15, NOAA-18, NOAA-19, S-NPP, Aqua, Terra, mixture of GEO/LEO	ACCESS-G3 uses METEOSAT-11 and GOES-16 which are not used by UKMO; the control and the experiment do not use these 2 satellites
Scatwind	MetOp-1, MetOp-2, CORIOLIS	MetOp-1, MetOp-2, CORIOLIS
GPSRO	MetOp-1, MetOp-2, TerraSAR-X, FY-3C	MetOp-1, MetOp-2, TerraSAR-X, COSMIC-1, COSMIC-6, FY-3C	N.B. Less satellites are used by ACCESS-G3
Ground GPS	used	used
MT/SAPHIR	not used	MT	From some time in Dec 2017 until 20180101T00 no data in obstore; from 20180101T06 obstores contain data
SEVIRI CSR	not used	METEOSAT-8, METEOSAT-10
AHI CSR	H-8	H-8
GOES CSR	not used	GOES-13, GOES-15

Note 1. For satwind and surface obstypes I could not come up with apps that would allow reading of all obs. For this I compared the numbers given by print-obstore with numbers in job.stats

Experimental set-up

For each TC case we will use a control and a test. The control will assimilate all observation types including those that we currently do not assimilation but are used by UKMO; the test will use only the observations used operationally in ACCESS-G3.

We will mainly assess the difference in track forecasts between control and test. However a question arises as to how robust the difference in track forecasts is. It's possible that the difference arose because of the inherent chaotic nature of the data assimilation system and the forecast model. To allow us to approximately the effect of this randomness we will also create an initial condition by adding global, random perturbations to the control.

For the trial period, 20180224T06-20180501T12 2 suites were set up:

• u-bl688 - control
• u-bm460 - experiment that excludes all observations which are not used by ACCESS-G3

For the trial period, 20190809T06-20190909T12 2 suites were set up:

• u-bm976 - control
• u-bn066 - experiment that excludes all observations which are not used by ACCESS-G3

Trial periods

The following is a list of Western Pacific TC's used in this experiment and the trial periods,

TC	First cycle of trial	last cycle of trial	lon/lat when first declared	Category	Comments
KAI-TAK	2017120406	2017122312
TEMBIN	2017121106	2017122512			straight track
HILDA	2017121706	2017122812
BOLAVEN	2017122406	2018010312
IRVING	2017122706	2018010612		Cat 1
JOYCE	2018010106	2018011312		Cat 1
CEBILE	2018011806	2018020812
FEHI	2018011906	2018013012
GITA	2018013106	2018022212		Cat 4
SANBA	2018020106	2018021612
KELVIN	2018020706	2018021912
HOLA	2018022406	2018031112		Cat 4	named on 20180306 and at its naming Cat 1
LINDA	2018030306	2018032512
MARCUS	2018030606	2018032512
NORA	2018030906	2018032712
IRIS	2018031406	2018040512
JELAWAT	2018031406	2018040112
JOSIE	2018032106	2018040212
KENI	2018032906	2018041112
FLAMBOYAN	2018041706	2018050112
EWINIAR	2018052306	2018060912

Starting cycletimes are chosen 10 days prior to the dates when TC's were declared ("Start date"). ???? might need to go a little further to allow analysis of pre-storm environment ?????

Tropical storms in the Atlantic Ocean over a similar period,

Tropical storm	First cycle of trial	last cycle of trial	lon/lat when first declared	Category	Comments
Tropical Storm Alberto					On 25 May, 2018 organisation sufficient to classify it as sub-tropical storm; 28 May it became a tropical storm and reached its peak intensity

Tropical cyclones for case study,

Tropical storm	First cycle of trial	last cycle of trial	lon/lat when first declared	Category	Comments
Dorian	20190809	20190909			19 Aug NHC identified a tropical wave in a monsoon trough over western Africa; Cat 5 on 1 Sep

Configuration

As one of the verification measures to be used in this experiment is the ability of the trials to predict TC tracks it is important to resolve the inner core structure of a TC; even better, to have a few gridpoints within the eye of a storm (Q. In order to model the interaction between a TC and its environment in related to TC movement do we need to resolve the eye?). To achieve this I would like to run UM at N640 resolution (see here for some issues encountered when changing the resolution of the ported PS41 suite from N320 to N640). However, running at this resolution for a trial lasting for more than 3 months is unaffordable at NCI. So as a compromise I decided to run at N320 UM resolution.

OPS

• Obstore files were retrieved from the update runs of UKMO operational OS,
  • all obstypes are tar'ed in a single tarball
  • raijin4:/g/data/dp9/as2291/obstores
  • 1.1 GB of data per cycle
  • from 20171204 00Z till 20180222 18Z (inclusive)

• Ops_CreateODB and Ops_ExtractAndProcess read observations from obstore files
  • Details on how to read obstore files and how to be sure all the observations from an obstore are read in correctly: wiki:ticket/370/TicketDetails/OpsReadFromObstore

• OPS control/data files
  • stationlist
    • thinning distance:

Obstype	Thinning distance - extratropics	Thinning distance - tropics
AHIClear	no thinning	no thinning
AIRS	125	154
AMSR	80	80
ATOVS	100	100
ATMS	125	154
CrIS	125	154
FY3B	125	154
GMIhigh	80	80
GMIlow	80	80
GOESClear	no thinning	no thinning
GPSRO	no thinning	no thinning
IASI	80	154
MTSAPHIR	no thinning	no thinning
MWSFY3C	80	80
Satwind	200	200
Scatwind	80	80
SEVIRIClear	no thinning	no thinning
SSMIS	100	154

17. How does the thinning distance interact with PFM resolution? For each obs Cx is a column interpolated horizontally and in time to the obs location. It is fixed in the inner loop. Cx+ = Cx + S^-I*(Cw) where Cw is dw interpolated horizontally and in time to the obs location in PFM grid. The observation operator then maps Cx+ to the observation space, y=V(Cx+) to give an estimate of the observed value.

VAR

• non-hybrid 4DVar
  • alternatively hybrid uncouple 4DVar with error modes from an archive of a single run

• PFM resolutions are N108 and N216

• used a new VAR build to get around a bug that incorrectly opens RTTOV coefficients - details here

UM

• Resolution: N320L70 (0.5625 deg lon and 0.375 deg lat; 63 km x 42 km at equator)
• PC2 cloud scheme
• ????

Results

Overall verification

While trial still running: ​GES output

• Score cards show the experiment run has degraded forecasts - blue dominates over green
• Reduced first-guess departures for the control,
  • IASI - more pronounced improvement around MetDB ch 120 which are mid- to lower tropospheric channels (weak, ozone channels), some window channels, around ch 270 (low-peaking, water vapour channels)
  • similar trend for other IR and MW instruments
• unexplained result is the obs count - in some cases the obs count for the control dropped (Q. Is this what I think it is?)

Note. Control (u-bl688) uses all the observations that UKMO use and the experiment (u-bm460) uses those observations that ACCESS-G3 uses

Verificaition of TC tracks

• TC tracker needs following fields,
  • essential: MSLP; 10-m winds; 850, 700, 500 hPa u/v winds and geopotential heights (available in the archived files, "*_glm_t???.ff")
  • additional: 800, 500, 300 hPa heights
  • fields need to be in Grib1 format
  • TC bogus files are needed to be used as first guess for the tracker
  • Jim to do an initial test of TC tracker once analysis and forecast files are available from a single basetime
    • test whether all STASH fields are output for TC tracker by cold-starting the suite from 20180306T06; long forecast from 20180306T12 should produce forecast files which can be used by the tracker

Resources

• ​Jim Fraser's List of TCs in West Pacific & Eastern Indian Ocean from G3 trial periods
  • Note "First Lat" and "First Lon" columns show lat/lon when TC's were first named; they are the TC locations on the dates under the column, "Start date"

• JTWC bogus central pressure observations are available from Dec, 2017
  • the central pressure data are converted to text files and then these text files are read in by OPS

Useful information

• Following table has different gridspacing used for various resolutions of UM and PFM

Resolution	dx x dy at the equator (deg)	dx x dy at the equator (km)
N108	1.6667 x 1.1111	185 x 124
N216	0.8333 x 0.5556	93 x 62
N320	0.5625 x 0.375	63 x 42
N640	0.3515625 x 0.234375	39 x 26

• The diammeter of the inner core of a typical tropical cyclone is ~300 km (Emanuel, 2018, Meteorological Monographs). So what is resolved for different resolutions is,
  • N320 - 5 gridpoints within the inner core, which means inner core is just resolved (using the "4 delta x" criterion)
  • N216 - 3 gridpoints within the inner core, which means inner core is not quite resolved
  • N108 - 2 gridpoints within the inner core, which means inner core is definitely not resolved

Diary of runs

u-bl688

Cycle time	Failed task	Reason for failure	Action taken
20180224T06 - 20180302T12	Not all batches in MTSAPHIR obstore files were read in; consequently ~1 in 10 observations was written out to varobs	In OPS namelist group, extractcontrolnl the variable, maxbatchessubtype was set too low	maxbatchessubtype changed to main app config value of 40
20180224T06 - 20180304T18	ODB2 files are not archived for this earlier time period	I didn't switch on archiving of ODB	ARCH_ODB2=True is used now
20180501T0600Z	gl[mu]_ops_process_background_seviriclear	SEVIRIClear obstore file contains METEOSAT-11 (SatId 70); as VarBC file does not contain parameters for this satellite the task fails	None since it is very nearly the end of the trial period, 20180501T12

u-bm460

Cycle time	Failed task	Reason for failure	Action taken
20180224T06 - 20180304T18	ODB2 files are not archived for this earlier time period	I didn't switch on archiving of ODB	ARCH_ODB2=True is used now
20180501T0600Z	gl[mu]_ops_process_background_seviriclear	SEVIRIClear obstore file contains METEOSAT-11 (SatId 70); as VarBC file does not contain parameters for this satellite the task fails	None since it is very nearly the end of the trial period, 20180501T12

u-bm976

Cycle time	Failed task	Reason for failure	Action taken
20190819T1200Z	glu_surf_ascat_ekf	Reading in Bufr file (?)	skipped all SURF tasks; ran glu_um_fcst task without updating soil moisture with glu_smc
20190902T1800Z	gl_ver_hk_ard	ARD_056_008_+7.00000E+02_T00000_G03_u-bm976-GM.con corrupted	Replaced the corrupted ARDfile with an archived copy; note the archive of ARD was done after all tasks of 20190902T00 so forecasts from 06Z, 12Z and 18Z are not in the ARDFile as well as missing verification stats for 06Z, 12Z and 18Z
20190904T1200Z	gl[mu]_ops_process_background_cris	the job failed when updating ODB1	reset to succeeded

u-bn066

Cycle time	Failed task	Reason for failure	Action taken
20190809T06	glu_um_fcst	NaN or Infinities developed	Did a FASTRUN
20190826T0000Z	glu_ops_process_analysis_satwind	Unknown	Set to succeeded
20190826T0600Z	glu_ops_process_background_cris	Unknown; varobs and varcx were created; exactly same error occurred to u-bm976 but it ran after retry	reset to succeeded; ToDo? Check if CrIS were used by VAR
20190826T0600Z	glu_ops_process_analysis_cris	Unknown	reset to succeeded
20190827T0600Z	glu_ops_process_analysis_ssmis	Same as what happen to glu_ops_process_analysis_cris.20190826T0600Z; TicketDetails/job.out	reset to succeeded
20190828T0000Z	gl[mu]_ops_process_background_cris	failure during updating of ODB1	reset to succeeded
20190828T0000Z	glu_ops_process_analysis_cris	All job.out has /home/548/jtl548/cylc-run/u-bn066/share/cycle/20190828T0000Z/glu_odb/cris is locked; because of the failure of glu_ops_process_background_cris while writing back to ODB1 .lock file might have been created	reset to succeeded
20190828T1200Z	gl_ops_ver_odb_obstore_satwind	Looks like error during opening of ODB1	reset to succeeded
20190828T1200Z	glu_ops_process_analysis_satwind	Looks like error during opening of ODB1	reset to succeeded

Things to do

• What do gl?_ops_process_analysis_* tasks do - generate feedback ODB1? Should these tasks be removed from the suite?

• TCBOGUS data seem to be available in UKMO obstores. Will they be used by VAR?

• What to do about locally received data?

• for gl_ops_process_satwind select based on satellite

• for gl_ops_process_gpsro select based on satellite

• set up 2 identical twins for a case study: e.g. Sandy, Irma or Dorian

• try using the following to fix the problem of job log output files not being retrieved,

  [hosts] -> [[HOST]] -> retrieve job logs max size
  [hosts] -> [[HOST]] -> retrieve job logs retry delays