22 Jan 2024
 | 22 Jan 2024

Decomposition of skill scores for conditional verification – Impact of AMO phases on the predictability of decadal temperature forecasts

Andy Richling, Jens Grieger, and Henning W. Rust

Abstract. We present a decomposition of skill scores for the conditional verification of weather and climate forecast systems. Aim is to evaluate the performance of such a system individually for predefined subsets with respect to the overall performance. The overall skill score is decomposed into: (1) the subset skill score assessing the performance of a forecast system compared to a reference system for a particular subset; (2) the frequency weighting accounting for varying subset size; (3) the reference weighting relating the performance of the reference system in the individual subsets to the performance of the full data set. The decomposition and its interpretation is exemplified using a synthetic data set. Subsequently we use it for a practical example from the field of decadal climate prediction: An evaluation of the Atlantic-European near-surface temperature forecast from the German initiative Mittelfristige Klimaprognosen (MiKlip) decadal prediction system conditional on different Atlantic Meridional Oscillation (AMO) phases during initialization. With respect to the chosen Western European North Atlantic sector, the decadal prediction system preop-dcpp-HR performs better than the un-initialized simulations mostly due to performance gain during a positive AMO phase. Compared to the predecessor system (preop-LR), no overall performance benefits are made in this region, but positive contributions are achieved for initialization in neutral AMO phases. Additionally, the decomposition reveals a strong imbalance among the subsets (defined by AMO phases) in terms of reference weighting allowing for sophisticated interpretation and conclusions. This skill score decomposition framework for conditional verification is a valuable tool to analyze the effect of physical processes on forecast performance and consequently supports model development and improvement of operational forecasts.

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.
Andy Richling, Jens Grieger, and Henning W. Rust

Status: final response (author comments only)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on egusphere-2023-2582', Jonas Bhend, 19 Feb 2024
  • RC2: 'Comment on egusphere-2023-2582', Anonymous Referee #2, 26 Feb 2024
Andy Richling, Jens Grieger, and Henning W. Rust
Andy Richling, Jens Grieger, and Henning W. Rust


Total article views: 257 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
194 48 15 257 11 10
  • HTML: 194
  • PDF: 48
  • XML: 15
  • Total: 257
  • BibTeX: 11
  • EndNote: 10
Views and downloads (calculated since 22 Jan 2024)
Cumulative views and downloads (calculated since 22 Jan 2024)

Viewed (geographical distribution)

Total article views: 258 (including HTML, PDF, and XML) Thereof 258 with geography defined and 0 with unknown origin.
Country # Views %
  • 1


Latest update: 12 Jun 2024
Short summary
The performance of weather and climate prediction systems is variable in time and space. It is of interest how this performance varies in different situations. We provide a decomposition of a skill score –a measure of forecast performance– as a tool for detailed assessment of performance variability to support model development or forecast improvement. The framework is exemplified with decadal forecasts to assess the impact of different ocean states in the North Atlantic on temperature forecast.