weather-lzone

Assimilation of PC-reconstructed radiances in NWP

 

weather-lzone
weather-lzone

This study, performed at ECMWF, shows that hyperspectral radiances reconstructed from principle components products can be seamlessly assimilated in an operational system designed for original radiances, preserving the nominal forecast performances. It was performed with IASI operational products, with a specific outlook on the preparation of MTG-IRS.

Last Updated

27 January 2023

Published on

08 June 2022

Hyperspectral missions have been providing key information to numerical weather prediction (NWP) for about two decades, e.g. from AIRS, IASI or CrIS instruments. The observations are disseminated to the users in the form of spectra made of thousands of channels. However, only small subsets of radiances (of the order of 100 channels) are actually assimilated in NWP models.

Principal Components Analysis enables high compression rates, while preserving the thermodynamic information and reducing the instrument noise in individual channels. The IASI principal components (PC) products from EUMETSAT achieve a compression factor of about 30 and have been operational since 2011. However, no operational experience has been established in NWP as users continued using the original radiances from the complete spectra, which were still disseminated in parallel.

The future geostationary sounder MTG-IRS will provide an unprecedented 4D look into the atmosphere. However, the data volume resulting from the unique spatial and temporal sampling demands that spectral information be compressed and disseminated in principal components scores. In this study, IASI is used as proxy for IRS to address a priori perceptions within the NWP community regarding the utilisation of principal components products.

The study performed at ECMWF shows that radiances reconstructed from PC products can be seamlessly ingested instead of original radiances in an operational assimilation system, with minimal adaptation. It also shows that the underlying eigenvector basis can be seamlessly updated over time without any system recalibration, ensuring operational continuity. The forecast’s performances with reconstructed radiances remain as high as with original radiances in all experiments. Therefore, principal components products form a plausible alternative route for the exploitation of high spectral resolution infrared sounders in NWP.


Objectives

The overarching question to be addressed was: Can an assimilation system set-up for conventional IASI radiances be switched to using reconstructed-radiances instead, without system retuning or forecast performance loss?

Study objectives:

  • Set up and run assimilation experiments of reconstructed radiances using different eigenvector configurations in the integrated forecast system (IFS) of ECMWF.
  • Evaluate the practicalities and impact of assimilating reconstructed, instead of original radiances by comparing the forecast skills against external references.
  • Evaluate operational continuity in case of an eigenvector basis update.
  • Share with the user community the conclusions and guidelines on the generation and utilisation of PC products for IRS.

Overview

System preparation and assimilation experiments definition
Radiance monitoring and cloud detection
Assimilation in a depleted environment
Assimilation in a full observing system
Assimilation run including an update of the eigenvector basis
Conclusions
Optional phase — Follow up on new methodologies to determine the observation error matrix

System preparation and assimilation experiments definition

In this study, two sets of PC products were used, resulting from two different eigenvector basis:

  1. v1.04 currently operational
  2. v2.01 to be released in the upcoming IASI L2 processor v6.7, second half of 2022.

The assimilation experiments with the original radiance, and the two sets of reconstructed radiances, first ran in a depleted observing system and then in a full observing system. They included observations from the three Metop platforms.

A pre-processing step to reconstruct the radiances from the principal components products was introduced. This was the only adaption made to the assimilation system. The same configuration as in normal operations with original radiances was used for the assimilation of reconstructed radiances. This includes the channel selection, variational bias correction (varBC), cloud screening and observation error.

  Depleted obs. system Full obs. system
Model CY47R1.4 TCo399 CY47R3.1 TCo399 (25km)
Period 09/2020-03/2021 02/06-31/08/2020
02/12/202-28/02/2021
CTRL Data in Figure 1, excluding IR sounders: 3 IASI, 2 CrIS, 1 AIRS Data in Figure 1, excluding 3 IASI
RAD L1C radiance, operational channel subset, varBC and observation error matrix.
REC_RAD_v104 Reconstructed radiances with eigenvector v1.04 (in operations).
Same channels, varBC and obs. error matrix as with original radiances.
REC_RAD_v201 Reconstructed radiances with eigenvector v2.01 (roll-out planned Q3-2022).
Same channels, varBC and obs. error matrix as with original radiances.
REC_RAD_COMB N/A Switching the eigenvector basis from v1.04 to v2.01 half-way through the assimilation experiments.
Overview of the observing system in ECMWF model
Figure 1: Overview of the observing system in ECMWF model

Radiance monitoring and cloud detection

  • The standard deviations of the background departures are reduced for temperature channels with reconstructed radiances, whereas they remain similar to statistics with original radiances for water vapour channels.
  • The biases are generally unchanged for all channels.
  • Comparable data volumes pass the cloud detection algorithm with original and reconstructed radiances.
Results from the monitoring of original and reconstructed radiances
Figure 2: Results from the monitoring of original and reconstructed radiances (left: standard deviation of observation minus background; right: bias correction), and radiance yield through the cloud detection (right). Global statistic for IASI Metop-A/B/C, 8 Sept–31 Dec 2020 and 8 Jan–30 April 2021.

Assimilation in a depleted environment

In the first phase, the assimilation experiments were carried out in a depleted environment.

Forecast skills in the original (green) and the reconstructed (red and black) radiance experiments vs the depleted Control (100%)
Figure 3: Forecast skills in the original (green) and the reconstructed (red and black) radiance experiments vs the depleted Control (100%). Temperature (left) and wind (middle) forecasts compared to radiosonde in situ measurements. On the right, 10-day forecast skills of the geopotential height at 500hPa in the southern (top) and northern (bottom) hemispheres.
  • Assimilation trials in a depleted system (no high-resolution IR sounder) suggest that reconstructed radiances (v1.04 and v2.01 baselines) can deliver as much thermodynamic information as IASI conventional radiances.
  • Both original and reconstructed radiance experiments produce similar forecast skill impact, and show statistically significant improvements over the depleted control.
  • No statistical difference in performance between the reconstructed experiments based on v1.04 and v2.01 eigenvectors bases.

Assimilation in a full observing system

The control run in the full observing system includes all observations available in nominal operations, including the infrared sounders, with the exception of IASI. It is expected that the impact of original and reconstructed radiances on the forecasts is less than in a depleted environment. But the relative impact of original and reconstructed radiances can be evaluated.

: Forecast skills in the original (green) and the reconstructed (red and black) radiance experiments vs the depleted Control (100%). Temperature (left), humidity (middle) and wind (right) forecasts compared to radiosonde in situ measurements.
Figure 4: Forecast skills in the original (green) and the reconstructed (red and black) radiance experiments vs the depleted Control (100%). Temperature (left), humidity (middle) and wind (right) forecasts compared to radiosonde in situ measurements.
Short-range forecast fit to satellite observations
Figure 5: Short-range forecast fit to satellite observations.
Mid-range forecast scorecards of reconstructed vs original experiments, verification against observation. Top: geopotential height at 500 hPa in northern and southern hemisphere. Bottom: temperature at 850 and 100 hPa.
Figure 6: Mid-range forecast scorecards of reconstructed vs original experiments, verification against observation. Top: geopotential height at 500hPa in northern and southern hemisphere. Bottom: temperature at 850 and 100hPa.
Forecast Sensitivity Observation Impact (FSOI) evaluated thourgh June-July-August 2020 with IASI original (top) and reconstructed (bottom) radiances, compared to other IR sounders and stratified against latitudinal bands (right)
Figure 7: Forecast Sensitivity Observation Impact (FSOI) evaluated thourgh June-July-August 2020 with IASI original (top) and reconstructed (bottom) radiances, compared to other IR sounders and stratified against latitudinal bands (right).

Assimilation run including an update of the eigenvector basis

In this last series of experiments, the eigenvectors basis of the PC products was updated ‘on-the-fly’, halfway through the assimilation period, while keeping all system configuration unchanged. The subsequent forecasts skills, having updated the eigenvector basis, are compared to assimilation experiments where the basis remained unchanged.

Forecast skills with reconstructed radiances, having updated (black) or left unchanged (red) the eigenvector basis through the assimilation experiments. Temperature (left), humidity (middle) and wind (right) forecasts compared to radiosonde in situ measurements.
Figure 8: Forecast skills with reconstructed radiances, having updated (black) or left unchanged (red) the eigenvector basis through the assimilation experiments. Temperature (left), humidity (middle) and wind (right) forecasts compared to radiosonde in situ measurements. Period: 02/06–31/08/2020 and 02/12/2020–28/02/2021.
Short-range forecast fit to satellite observations
Figure 9: Same as above, short-range forecast fit to satellite observations.

Conclusions

✔ No loss of performance switching from original to reconstructed radiances in a depleted environment

  • Assimilation of IASI reconstructed radiances working with minimal adaptation in the IFS.
  • Positive impact on analyses and forecasts, like the conventional IASI radiances.
  • Very promising results using the current dedicated operational observation errors for IASI radiances.
  • Plausible route for the exploitation of high-resolution IR data in NWP.

✔ No indications of performance loss switching from RAD to REC_RAD in the full system.

  • Full system experiments conducted to assess REC_RAD impacts when analysis errors are small.
  • Positive impact on analyses and forecasts, similar to RAD; Short and medium-range impact differences between REC_RAD systems and RAD are not statistically significant.
  • OSEs assessment was complemented by an analysis of Forecast Sensitivity to Observation Impact diagnostics.

✔ No indications of performance loss switching between REC_RAD eigenvectors basis, ensuring operational continuity.

  • REC_RAD eigenvectors base was changed 'on-the-fly' from v.104 to v2.01 to simulate a necessary update in an operational context.
  • Impact on analyses/forecasts is similar to RAD.

Optional phase — Investigating new methodologies to derive suitable observation error matrices for new instruments with spectral correlation

The success of assimilating reconstructed radiances was also permitted by the pre-existing suitable, well-tuned observation error matrix for IASI radiances.

Further work on the determination of a functional observation error matrix would be very beneficial for any new instrument data presenting error correlations. The problem is not specific to reconstructed radiances, whose operational potential was shown in the first phase.

Therefore, an additional exploration has been undertaken to accelerate the generation of a functional observation error matrix and hence accelerate the operational uptake of MTG-IRS in NWP after commissioning. This includes exploring parametric approaches and investigating the observation and model subspaces.