Preprints
https://doi.org/10.5194/egusphere-2023-2582
https://doi.org/10.5194/egusphere-2023-2582
22 Jan 2024
 | 22 Jan 2024

Decomposition of skill scores for conditional verification – Impact of AMO phases on the predictability of decadal temperature forecasts

Andy Richling, Jens Grieger, and Henning W. Rust

Abstract. We present a decomposition of skill scores for the conditional verification of weather and climate forecast systems. Aim is to evaluate the performance of such a system individually for predefined subsets with respect to the overall performance. The overall skill score is decomposed into: (1) the subset skill score assessing the performance of a forecast system compared to a reference system for a particular subset; (2) the frequency weighting accounting for varying subset size; (3) the reference weighting relating the performance of the reference system in the individual subsets to the performance of the full data set. The decomposition and its interpretation is exemplified using a synthetic data set. Subsequently we use it for a practical example from the field of decadal climate prediction: An evaluation of the Atlantic-European near-surface temperature forecast from the German initiative Mittelfristige Klimaprognosen (MiKlip) decadal prediction system conditional on different Atlantic Meridional Oscillation (AMO) phases during initialization. With respect to the chosen Western European North Atlantic sector, the decadal prediction system preop-dcpp-HR performs better than the un-initialized simulations mostly due to performance gain during a positive AMO phase. Compared to the predecessor system (preop-LR), no overall performance benefits are made in this region, but positive contributions are achieved for initialization in neutral AMO phases. Additionally, the decomposition reveals a strong imbalance among the subsets (defined by AMO phases) in terms of reference weighting allowing for sophisticated interpretation and conclusions. This skill score decomposition framework for conditional verification is a valuable tool to analyze the effect of physical processes on forecast performance and consequently supports model development and improvement of operational forecasts.

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.

Journal article(s) based on this preprint

22 Jan 2025
Decomposition of skill scores for conditional verification: impact of Atlantic Multidecadal Oscillation phases on the predictability of decadal temperature forecasts
Andy Richling, Jens Grieger, and Henning W. Rust
Geosci. Model Dev., 18, 361–375, https://doi.org/10.5194/gmd-18-361-2025,https://doi.org/10.5194/gmd-18-361-2025, 2025
Short summary
Andy Richling, Jens Grieger, and Henning W. Rust

Interactive discussion

Status: closed

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on egusphere-2023-2582', Jonas Bhend, 19 Feb 2024
  • RC2: 'Comment on egusphere-2023-2582', Anonymous Referee #2, 26 Feb 2024
  • AC1: 'Comment on egusphere-2023-2582', Andy Richling, 01 Aug 2024

Interactive discussion

Status: closed

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on egusphere-2023-2582', Jonas Bhend, 19 Feb 2024
  • RC2: 'Comment on egusphere-2023-2582', Anonymous Referee #2, 26 Feb 2024
  • AC1: 'Comment on egusphere-2023-2582', Andy Richling, 01 Aug 2024

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision | EF: Editorial file upload
AR by Andy Richling on behalf of the Authors (02 Aug 2024)  Author's response   Author's tracked changes   Manuscript 
ED: Referee Nomination & Report Request started (26 Aug 2024) by Sophie Valcke
RR by Jonas Bhend (06 Sep 2024)
RR by Anonymous Referee #2 (18 Sep 2024)
ED: Publish subject to technical corrections (07 Oct 2024) by Sophie Valcke
AR by Andy Richling on behalf of the Authors (11 Oct 2024)  Author's response   Manuscript 

Journal article(s) based on this preprint

22 Jan 2025
Decomposition of skill scores for conditional verification: impact of Atlantic Multidecadal Oscillation phases on the predictability of decadal temperature forecasts
Andy Richling, Jens Grieger, and Henning W. Rust
Geosci. Model Dev., 18, 361–375, https://doi.org/10.5194/gmd-18-361-2025,https://doi.org/10.5194/gmd-18-361-2025, 2025
Short summary
Andy Richling, Jens Grieger, and Henning W. Rust
Andy Richling, Jens Grieger, and Henning W. Rust

Viewed

Total article views: 533 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
353 151 29 533 23 22
  • HTML: 353
  • PDF: 151
  • XML: 29
  • Total: 533
  • BibTeX: 23
  • EndNote: 22
Views and downloads (calculated since 22 Jan 2024)
Cumulative views and downloads (calculated since 22 Jan 2024)

Viewed (geographical distribution)

Total article views: 524 (including HTML, PDF, and XML) Thereof 524 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 

Cited

Latest update: 22 Jan 2025
Download

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Short summary
The performance of weather and climate prediction systems is variable in time and space. It is of interest how this performance varies in different situations. We provide a decomposition of a skill score –a measure of forecast performance– as a tool for detailed assessment of performance variability to support model development or forecast improvement. The framework is exemplified with decadal forecasts to assess the impact of different ocean states in the North Atlantic on temperature forecast.