Preprints
https://doi.org/10.5194/egusphere-2023-2720
https://doi.org/10.5194/egusphere-2023-2720
24 Nov 2023
 | 24 Nov 2023

Objective Evaluation of Earth System Models: PCMDI Metrics Package (PMP) version 3

Jiwoo Lee, Peter J. Gleckler, Min-Seop Ahn, Ana Ordonez, Paul A. Ullrich, Kenneth R. Sperber, Karl E. Taylor, Yann Y. Planton, Eric Guilyardi, Paul Durack, Celine Bonfils, Mark D. Zelinka, Li-Wei Chao, Bo Dong, Charles Doutriaux, Chengzhu Zhang, Tom Vo, Jason Boutte, Michael F. Wehner, Angeline G. Pendergrass, Daehyun Kim, Zeyu Xue, Andrew T. Wittenberg, and John Krasting

Abstract. Systematic, routine, and comprehensive evaluation of Earth System Models (ESMs) facilitates benchmarking improvement across model generations and identifying the strengths and weaknesses of different model configurations. By gauging the consistency between models and observations, this endeavor is becoming increasingly necessary to objectively synthesize thousands of simulations contributed to the Coupled Model Intercomparison Project (CMIP) to date. The PCMDI Metrics Package (PMP) is an open-source Python software package that provides "quick-look" objective comparisons of ESMs with one another and with observations. The comparisons include metrics of large- to global-scale climatologies, tropical inter-annual and intra-seasonal variability modes such as El Niño-Southern Oscillation (ENSO) and Madden-Julian Oscillation (MJO), extratropical modes of variability, regional monsoons, cloud radiative feedbacks, and high-frequency characteristics of simulated precipitation, including extremes. The PMP results are produced in the context of all model simulations contributed to CMIP6 and earlier CMIP phases. An important priority of the PMP is to document evaluation statistics for all Historical and AMIP simulations submitted to recent phases of CMIP, providing version-controlled information for all data sets and software packages being used. Among other purposes, this also enables modeling groups to assess performance changes during the ESM development cycle in the context of the error distribution of the multi-model ensemble. In this paper, we present an overview of the PMP including its history to date, capabilities, recent updates, and future direction.

Jiwoo Lee, Peter J. Gleckler, Min-Seop Ahn, Ana Ordonez, Paul A. Ullrich, Kenneth R. Sperber, Karl E. Taylor, Yann Y. Planton, Eric Guilyardi, Paul Durack, Celine Bonfils, Mark D. Zelinka, Li-Wei Chao, Bo Dong, Charles Doutriaux, Chengzhu Zhang, Tom Vo, Jason Boutte, Michael F. Wehner, Angeline G. Pendergrass, Daehyun Kim, Zeyu Xue, Andrew T. Wittenberg, and John Krasting

Status: final response (author comments only)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on egusphere-2023-2720', Anonymous Referee #1, 04 Dec 2023
  • RC2: 'Comment on egusphere-2023-2720', Anonymous Referee #2, 18 Dec 2023
Jiwoo Lee, Peter J. Gleckler, Min-Seop Ahn, Ana Ordonez, Paul A. Ullrich, Kenneth R. Sperber, Karl E. Taylor, Yann Y. Planton, Eric Guilyardi, Paul Durack, Celine Bonfils, Mark D. Zelinka, Li-Wei Chao, Bo Dong, Charles Doutriaux, Chengzhu Zhang, Tom Vo, Jason Boutte, Michael F. Wehner, Angeline G. Pendergrass, Daehyun Kim, Zeyu Xue, Andrew T. Wittenberg, and John Krasting
Jiwoo Lee, Peter J. Gleckler, Min-Seop Ahn, Ana Ordonez, Paul A. Ullrich, Kenneth R. Sperber, Karl E. Taylor, Yann Y. Planton, Eric Guilyardi, Paul Durack, Celine Bonfils, Mark D. Zelinka, Li-Wei Chao, Bo Dong, Charles Doutriaux, Chengzhu Zhang, Tom Vo, Jason Boutte, Michael F. Wehner, Angeline G. Pendergrass, Daehyun Kim, Zeyu Xue, Andrew T. Wittenberg, and John Krasting

Viewed

Total article views: 296 (including HTML, PDF, and XML)
HTML PDF XML Total Supplement BibTeX EndNote
188 95 13 296 31 12 8
  • HTML: 188
  • PDF: 95
  • XML: 13
  • Total: 296
  • Supplement: 31
  • BibTeX: 12
  • EndNote: 8
Views and downloads (calculated since 24 Nov 2023)
Cumulative views and downloads (calculated since 24 Nov 2023)

Viewed (geographical distribution)

Total article views: 286 (including HTML, PDF, and XML) Thereof 286 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 26 Feb 2024
Download
Short summary
We introduce an open-source software, the PCMDI Metrics Package (PMP), developed for a comprehensive comparison of Earth System Models (ESMs) with real-world observations. Using diverse metrics evaluating climatology, variability, and extremes simulated in thousands of simulations from the Coupled Model Intercomparison Project (CMIP), PMP aids in benchmarking model improvements across generations. PMP also enables efficient tracking of performance evolutions during ESM developments.