Version 35 (modified by Jin Lee, 3 years ago) (diff)


Assimilating Himawari-8/AHI clear-sky radiances in APS3 global suite


  1. Single observation test - look at vertical and horizontal spread of an CSR radiance
  2. Effect of AHI CSR radiances on 4DVAR convergence
  3. "OSDP 6 - SatRad processing of geostationary clear-sky radiances" reports,
    1. "Improvement in analysed winds and humidities between 400 and 800 hPa has been observed."
    2. "how well it [analysis] is fitted by moisture-sensitive channels on other instruments: this has been found to be improved in more cases than not."

These 2 findings will be tested in our configuration

Experimental Set-up


Suites used

Following 2 suites were used,

Type Suite Summary of changes Comment
Control u-aj730 A copy of the standard APS3 global suite, u-ag312@24892 (equivalent to UKMO PS38 suite).

Note. At the particular revision when the suite was copied the trial period was 20160515T06 - 20160731T12. This suits my purpose as H8/AHI CSR data are available from marsdev starting from May 2016 (???? when????)
Single-obs trial u-aj977 a copy of u-aj730 with modifications to make OPS tasks for AHI CSR work; further modfications to assimilate only a single obs
Longer trial u-am137 a copy of u-aj730 with modifications to make OPS tasks for AHI CSR work; copy of u-aj977 at an earlier revision plus additional mods

Note 1. u-ag312 differs from PS38 suite, u-ad365: see rose-suite.conf

Note 2. There was an error introduced to u-ag312, which propagated to all child suites including u-aj730 and u-aj977. The error was the use of PS37 initial VarBC file instead of PS38 one, which meant the coefficients for FY-3B and Himawari-8 were missing. The 2 suites were updated to use PS38 initial VarBC file

OPS build used

My development OPS branch is r3192_810_ahicsr_bom_bufr

  • Before making any changes I built this and did a quick test to see if it produces same Aircraft varobs. The Aircraft varobs files from this build is identical to the build, '/projects/access/nwpdir/share/APS3/OPS/ops-2016.03.0'.
  • See here for OPS code changes to make Ops_CreateODB correctly do the Bufr-to-ODB conversion of Bufr files received from MSC

Turning on gl_ops_process_ahiclear app and related apps

NCI optional app config created to replace data extraction from MetDB (used by UKMO) with a direction conversion of HIMCSR Bufr files to ODB databases.

Impact of a Single Observation

Selection of clear AHI CSR segments

To obtain complete information about each and every "FOV" (JMA/MSC uses the term, "segment" because each channel observation OPS processes is an aggregation of 16x16 pixels) SatRad NetCDF writefile was turned on.

To reduce the amount of data read in by OPS only a single split HIMCSR Bufr file for 20160515T06 was used.

From the writefile a single clear FOV was selected based on QCflags and its lat/lon noted,


Then filtering was applied using "extractcontrolnl{ahiclear}" namelist of "gl_ops_process_ahiclear" app config by specifying geographic bounds in order to reduce the number of FOVs input to OPS to about a dozen. Following setting allowed 4 FOVs to be passed to varobs file,


I refined the extractcontol namelist to allow a single FOV to be passed to varobs using the following,


Next I modified channel selection file to allow only a single channel observation to be written out to varobs????

For comparison I did a single-observation test using IASI (see here)

Longer trial

  • I modified u-am137 to archive ODB2 and SatRad NetCDF writefiles for ahiclear.
  • Trial period is from 20160515T06 till 20160611T00 (4 weeks; 1 week of spin-up followed by 3 weeks of clean trial)


Impact of a Single Observation

The vertical levels where the analysis increments are largest do not coincide with the peak heights of weighting. To better understand the relationship between the 2 try following experiments:

  • 3DVAR - this eliminates the effect of PF and its adjoint
  • non-hybrid - this tests the spreading of observational information by static covariance
  • Try assimilating channel obs from IASI - weighting functions of IASI channels are sharper so may be easier to interpret

Note 1. ensemble covariance may not be used at higher levels in UKMO hybrid VAR

Longer trial


Cycle time Failed task Reason for failure Action taken
20160523T0000Z glm_var_anal_n216 stdout and stderr
20160524T1200Z glm_um_recon_em_n108, glm_um_recon_em_n216 My disk quota exceeded and the suite was in a strange state; when I reran the tasks ensemble forecast files from previous cycle were cleaned out and these tasks failed. Looking at the archive it looks as though all the cycles up to 20160524T18 ran and the current cycle is 20160525T00 Reset succeeded all cycles up to 20160524T18 and continued with 20160525T00
20160526T06 engl_ens_addssts_031 stdout has a message,

WARNING: Merging SST Perts with ETKF perts has not worked for member 031.

stderr has,

/home/548/jtl548/cylc-run/u-aj730/share/fcm_make_var-opt/build-serial/bin/VarScr_UMFileUtils: line 296: 1100: Bus error

perturbed SST field seems to be in the analysis perturbation so triggered task again and it succeeds
Cycle time Failed task Reason for failure Action taken
20160516T18 glu_var_anal_n108 stderr log says PF_bdy_lyr.f90 failed; at niter= 25 something went wrong; it appears for this cycle mu seems negative more often than other cycles during inner-loop iteration but not sure if this is the cause of the failure reran task and it succeeded; compared stdout log outputs from previous, failed job and from the successful run and the numbers are exactly same until niter=25

Useful information

  • In the suite the OPS tasks for H-8 AHI CSR use the label, 'ahiclear'
  • In the OPS source code the obsgroup used for H-8 AHI CSR is 'ObsGroupAHIClr' - see 'OpsMod_ObsInfo/OpsMod_ObsGroupInfo.f90'
  • In the OPS source code the MetDB subtype used for AHI CSR is 'HIMCSR' - see '../public/Ops_Constants/'


  1. In raijin4:/g/data/dp9/da/access-g/ops/bufr add "ahiclear.*.bufr" to ECMA tarballs
  1. In "gl[um]_ops_process_background_ahiclear" tasks may need to modify MetDB elements file
    • the repetition may not match what's in ODB