Almost 200 experts from around the world joined a workshop on the treatment of random and systematic errors in satellite data assimilation for numerical weather prediction (NWP), organised by ECMWF and the NWP SAF and held online from 2 to 5 November 2020.
The meeting continued the strong tradition of workshops organised jointly with the NWP SAF, aimed at bringing experts together to exchange ideas and views on pertinent topics. The NWP SAF is funded by EUMETSAT, and a key aim is to develop tools and services to support the advanced operational exploitation of satellite data in NWP.
Dealing with random and systematic errors in observations and models is at the heart of making optimal use of the wealth of information from satellite data to initialise forecasts or to create reanalyses. Errors and uncertainties arise from many areas, such as in the forecast model, the observations, or the ‘observation operators’ used to convert model fields to observation equivalents. The challenge is to separate the different errors and to deal with them adequately during the assimilation.
The 23 speakers reviewed the strong progress in recent years in several areas, both in the way errors are treated in the assimilation as well as in the understanding of the characteristics of the errors.
One of the aims of the workshop was to connect activities in different communities, spanning data providers, operational forecasting centres and academia, and to identify where NWP can make use of advances made in other fields in the characterisation of observation-related uncertainties. Reanalysis featured as a strong component, where treating biases in observations and models poses its own challenges.
“There were strong interactions between participants around the globe, in no small part thanks to the huge dedication shown by some participants who stayed up late or got up early to join their European colleagues live in rich exchanges through online chats and video-conferencing rooms,” said Niels Bormann, one of the organisers. Poster sessions, panel discussions and working groups provided further opportunities for discussion.
Panel discussions on the treatment of biases and the treatment of random observation errors considered the main challenges and priorities going forward, feeding into lively working group discussions.
Characterisation of uncertainties in observations and models
Knowledge about observational and model errors and uncertainties informs our treatment of these in the assimilation. On the topic of instrumental biases, talks covered the latest developments in the on-orbit characterisation of the CrIS hyperspectral infrared instruments and the Aeolus Doppler Wind Lidar, as well as the ongoing activities of the Global Space-Based Inter-calibration System (GSICS, a WMO/CGMS initiative).
The development of metrology-inspired approaches to the characterisation of observational uncertainties featured prominently in two of the talks, and the topic was revisited frequently in the panel discussion on biases and in working group discussions. “This was one of the areas where the aim of exploring new approaches from communities outside of the field of NWP data assimilation was met successfully,” says Bill Bell, one of the organisers.
Michael Rennie (ECMWF) showed the latest work on a bias model developed for the Aeolus Doppler Wind Lidar, using measurements of the main telescope mirror temperatures to correct for instrument effects.
Representation error, resulting from a mismatch between the model representation of the state and observations, was explored in two of the talks and promising approaches were described involving high-resolution model runs and observations. A talk on the characterisation of uncertainties in historic observations illustrated the value of the long-term perspectives brought by reanalyses and set the scene for how these are used in reanalysis systems in subsequent talks.
Paul Poli (EUMETSAT) discussed uncertainty analyses for historical datasets and their potential use in reanalyses.
Correction of observational and model biases in data assimilation
Correcting observational biases is essential for the successful assimilation of many satellite observations. Adaptive bias correction methods are now commonly used within the assimilation system. They are based on the specification of observation bias models that remove the systematic error when the observation departures are computed. Several talks discussed the best way to build these observation bias models to avoid absorbing model error in this correction (e.g. Constrained VarBC, anchor-only reference state, better choice of predictors).
Mark Buehner (Environment and Climate Change Canada) produced a 3D-Var analysis assimilating only anchor observations that can be used in the observation bias model of the main 4D-EnVar system to reduce the bias in the analysis and the short-term forecasts.
Separating observation and model biases continues to be challenging. Several presentations discussed the concept of scale separation where model error contains identifiable large-scale structures, which opens a new perspective in the quest to attribute the correct source of biases. Machine learning methods trained on analysis increments or anchor observations also showed some potential to learn the correct structure of model biases.
All the talks illustrated the importance of anchor observations and more specifically GNSS RO to estimate the different types of biases. In this context, reanalysis is facing a challenge, with highly variable observational coverage that becomes sparse going further back in time.
Representation of observation errors in data assimilation
The random error characteristics assigned to observations in the assimilation play a key role in determining the weighting of observations in the assimilation. Aside from measurement errors, many satellite observations are also strongly affected by representation error, that is, errors that occur when model fields are mapped into observation equivalents. These are often situation dependent and correlated between different observations.
Both aspects are increasingly being taken into account in operational systems. The required observation error modelling mostly relies on diagnostics based on differences between observations and model equivalents from short-range forecasts or analyses, and several speakers highlighted the potential, but also pitfalls of these approaches.
In the case of situation-dependent models, results from the diagnostics are used to determine simple parametrized models that capture main variations in error contributions, for instance, from poorer surface or cloud modelling. The use of inter-channel error correlations is now widespread, and concepts for modelling situation-dependent inter-channel error correlations are being developed. A better understanding and treatment of spatially correlated observation errors is also emerging, though algorithmic challenges remain, especially for variational data assimilation systems.
The panel discussion and working groups identified further refinements in observation error modelling as key aspects for progress (such as further enhanced situation-dependent observation error modelling, accounting for horizontal error correlations). They also emphasised the benefit of better independent characterisation of all error contributions (including representation error) and the need for further work in this area.
Fiona Smith (BoM) reported on the status of accounting for inter-channel error correlations for hyperspectral IR instruments at leading NWP centres, highlighting similarities and differences in their treatment. Inter-channel error correlations are now used by all major centres, leading to stronger forecast impact of the observations.
Looking to the future
A growing and increasingly diverse observing system as well as developments in assimilation methods will pose additional challenges for the treatment of systematic and random errors. Hyperspectral sounding data with unprecedented temporal resolution will be available from geostationary satellites, observations from some smaller satellites may have less well-characterised uncertainties, and an increasing number of observations will be driving multiple Earth system components with different model error and levels of maturity. All of these aspects will require continued development of the methods and capabilities used, including for NWP-independent characterisation of all error sources.
Further information
Recordings of all presentations and panel discussions are available on the workshop web page, as well as the posters presented. A workshop report with outcomes from the working groups will be published on the workshop web page in due course.