Preprints
https://doi.org/10.5194/egusphere-2026-5469
https://doi.org/10.5194/egusphere-2026-5469
16 Nov 2026
 | 16 Nov 2026
Status: this preprint is open for discussion and under review for Geoscientific Model Development (GMD).

Advancing climate model evaluation and intercomparison: an error decomposition-based index to quantify statistical sources of climate model errors

Meng-Zhuo Zhang, Zhongfeng Xu, Ying Han, and Weidong Guo

Abstract. Root Mean Square Error (RMSE) and Kling-Gupta Efficiency (KGE) are widely used performance metrics for quantifying the mismatch between simulated and observed data. They can be expressed in terms of three components: mean error, anomaly amplitude error, and anomaly similarity error. This decomposition makes it possible to identify the primary error source affecting overall model performance. However, the RMSE does not vary monotonically with model performance, which may mislead the evaluation results, while KGE is insensitive to anomaly similarity error, especially when the correlation coefficient value is high. To address these shortcomings, we proposed a novel error-decomposition index, termed the Decomposable Error Index (DEI), for measuring overall model error. The DEI is built from individual penalty terms that target mean, anomaly amplitude, and anomaly similarity errors. When the other two errors are held constant, the DEI varies monotonically with each of these three errors, thereby avoiding the non-monotonic limitation of RMSE. Moreover, the DEI imposes stronger penalties on the anomaly similarity error than KGE does. The DEI error decomposition statistically identifies the primary error source of the overall model error and pinpoints the primary driver for the inter-model performance difference. Beyond the evaluation of a scalar field, we also extended DEI and its decomposition into vector or multivariate fields. Applying DEI and its decomposition to evaluate CMIP models over East Asia, we identified the anomaly similarity error as the primary source of the overall performance for precipitation and multi-level winds across 26 CMIP6 models in both winter and summer, explaining 42–99 % of DEI2. It underscores the priority of improving pattern similarity for their simulations in future model development. Specifically, the pattern similarity improvement in wind and summer precipitation can significantly benefit from enhancing the horizontal resolution. In evaluating generational progress of 13 CMIP5-6 model pairs, the DEI decomposition pinpoints the reduced anomaly similarity error (mean error) as the key driver responsible for the improved overall performance of multi-level wind simulations in winter and summer (multi-level air temperature simulation in summer) over this region, with over 53 % contribution.

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors. Views expressed in the text are those of the authors and do not necessarily reflect the views of the publisher.
Share
Meng-Zhuo Zhang, Zhongfeng Xu, Ying Han, and Weidong Guo

Status: open (until 11 Jan 2027)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
Meng-Zhuo Zhang, Zhongfeng Xu, Ying Han, and Weidong Guo
Meng-Zhuo Zhang, Zhongfeng Xu, Ying Han, and Weidong Guo

Viewed

Total article views: 47 (including HTML, PDF, and XML)
HTML PDF XML Total Supplement BibTeX EndNote
39 4 4 47 7 9 10
  • HTML: 39
  • PDF: 4
  • XML: 4
  • Total: 47
  • Supplement: 7
  • BibTeX: 9
  • EndNote: 10
Views and downloads (calculated since 16 Nov 2026)
Cumulative views and downloads (calculated since 16 Nov 2026)

Viewed (geographical distribution)

Total article views: 47 (including HTML, PDF, and XML) Thereof 47 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 17 Nov 2026
Download
Short summary
We proposed the Decomposable Error Index (DEI), a novel error index indicating the overall performance of climate models. DEI is built from three individual penalty terms that target mean, anomaly amplitude, and anomaly similarity errors, respectively. By quantifying the contribution of each statistical error to DEI, this method can identify the primary error source of the overall model error and pinpoint the primary driver of the inter-model performance difference.
Share