the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 14 Jan 2025
The Coupled Model Intercomparison Project (CMIP): Reviewing project history, evolution, infrastructure and implementation
Abstract. The CMIP6 project was the most expansive and ambitious Model Intercomparison Project (MIP), the latest in a long history, extending back four decades. CMIP has captivated and engaged a broad, growing community focused on improving our climate understanding. It has anchored our ability to quantify and attribute the drivers and responses of the observed climate changes we are experiencing today.
The project's profound impact has been achieved by combining the latest climate science and technology. This has enabled the production of latest-generation climate simulations and the dissemination of their output, which has seen increased community attention in every successive phase. The review emphasizes the pragmatics of progressively scaling up efforts, the evolution of how the MIPs were implemented, and the coordinated efforts to establish a minimal infrastructure to make that possible, most recently delivering CMIP6.
- Preprint
(1390 KB) - Metadata XML
- BibTeX
- EndNote
Status: open (until 11 Mar 2025)
-
CC1: 'Comment on egusphere-2024-3729', Bjorn Stevens, 20 Jan 2025
reply
Publisher’s note: the content of this comment was removed on 20 January 2025 since the comment was posted by mistake.
Citation: https://doi.org/10.5194/egusphere-2024-3729-CC1 -
CC2: 'Comment on egusphere-2024-3729', Bjorn Stevens, 20 Jan 2025
reply
The paper provides a wealth of information about the mechanics of CMIP and in this sense is a very useful documentation of these activities and the present status by many of the key figures involved. The summary of the provision and usage of CMIP data and development of the infrasturcute was particularly useful and novel. While, as the paper’s introduction states, many aspects of CMIPs history have been shared before, by many of the same authors, this telling is more comprehensive and serves its stated purpose of providing a uniform treatment. From a more critical perspective the paper wades into value judgements that it fails to substantiate, when these are combined with its overall uncritical outlook, it comes across as needlessly self congratulatory, which detracts from its more meritorious contributions.
A defining quality of scientific work, e.g., as described by, the Mertonian norms, is organized skepticism. The present contribution would be strengthened if this were in greater evidence. For instance the articles by myself and Jakob et al. are dismissed in passing, at the end, without addressing the question they raise, which is whether the continued growth of CMIP as a quasi operational activity (Meehl’s words) that uses a research, rather than an operational, infrastructure is good for research or helpful for society. By binding research to a quasi operational activity what burden does CMIP place on free an innovative research? How effective is it in addressing feedback from downstream users (consider that climate services in Africa are still based on downscaling of CMIP5 data), e.g., Jakob et al? Why isn’t it leading to markedly better models? Can the growing demands on the infrastructure continue to be borne by the research community or does it need a more permanent footing, as was, for instance advocated for by EVE.
Relatedly, the manuscript makes many unqualified and unsupported statements about the impact and success of CMIP. Consider the statement (l1082): "CMIP has generated profound scientific insights that define how we understand and address climate change and our ability to quantify and attribute the drivers and responses to the observed climate changes we are experiencing today." Related to this is the statement in the abstract that "CMIP has captivated and engaged a broad, growing international community focused on improv-ing our climate understanding" or reference to the CMIP project’s central goal "to advance scientific understanding of the Earth system and its responses to ongoing natural and anthropogenic forcing agents" (line 349) or most grandiosely "However, without CMIP, the IPCC assessments could not have been possible. Without the coordinated community climate science efforts embodied by the AMIP and CMIP phases, progress in Earth’s climate understanding would not have advanced to our present state of knowledge."
The manuscript clearly demonstrates that there is a lot of CMIP data and that this data is widely used. This is certainly one measure of impact. I also think the progression in understanding through the first three CMIP phases is reasonably well documented. But with much of the growth occurring after CMIP3 the critical question is what — scientifically — has been wrought of the additional effort. There is a common perception, which I share, that what was to be learned from CMIP was mostly learned in CMIP3, and CMIP5’s contribution was to confirm that. Given the growth between CMIP5 and CMIP6, and its associated cost, what is the incremental gain? Certainly updating the scenarios is a necessary and valuable activity, but is it well served by the uncertain and adhoc time-lines, and does it need to be done through an activities like CMIP. The manuscript doesn't answer these questions. It doesn't have to answer these question, unless it wants to continue to attach value to the mechanics it describes. And in that case it must do so in a systematic and critical way. This requires addressing some of the arguments in my essay, for instance why some of the biggest and most lauded steps forward happened in areas where model output was disregarded, as for the case of climate sensitivity, and why there is little evidence of model improvement, beyond the null hypothesis that nominal gains are from improved resolution and gaming the diagnostics.
Even if the paper limits itself to a discussion of the mechanics of CMIP,. rather than a valuation of these mechanics, a skeptical point of view would be refreshing. What hasn't worked? What should be done differently even if the authors are of the mind that CMIP should be an activity that remains a service of the research community, i.e., is pursued using a research infrastructure. Has the considerable investments in automated evaluation tools, reprocessing of satellite data to align with model output, and model documentation activities really been beneficial? If so this must be demonstrate rather than just claimed.
One disadvantage of large author lists where most of the author’s don’t meaningfully contribute, is that it makes it difficult to develop a critical outlook, and this is certainly the case here. It also undermines good scientific practice. Surely the standard for authorship of a scientific paper requires more than having "contributed to the final version of the manuscript." Considering how fewer than half the authors are recognized for specific contributions, even contributions as small as commenting on a section, it makes the contributions of the non listed authors seem even more trivial. Given that many people who are not authors contributed to the design and execution of various phases of CMIP, not to mention the many modelling groups, a higher bar on authorship seems warranted.
Rather more editorial comments:
As a minor and historical note, when discussing the DECK, I suggested it as a component of Meehl's (as best I can recall) caricature of a CMIP6 prototype. If my memory serves me well Meehl proposed a prototype version of CMIP6 as a sailing ship of discovery, with different sails representing different MIPS. Based on this I proposed the DECK as the place that secured the masts for the sails, and as such as a specific response to an earlier (but unreferenced) criticism of CMIP by Marotzke and Rauser. The critique being that CMIP phases lacked continuity. This was an example of how CMIP incorporated and adapted to criticism.
In many places this reader gets the feel that quantity is a measure of quality. This works for describing water, but not for scientific advancements
The authors may want to guard against the impression, which is evident at the end, even though it is dealt with much better earlier in the manuscript, that somehow model intercomparison is an activity that started with AMIP.
I am not sure what the authors have in mind when they refer to CMIP6 models having fewer parameterizations (l570). Maybe this could be made explicit.
-- Bjorn Stevens
Citation: https://doi.org/10.5194/egusphere-2024-3729-CC2 -
CC3: 'Comment on egusphere-2024-3729', Rasmus Benestad, 06 Feb 2025
reply
Thank you for this account of the CMIP and its history. It’s very nice to get such a detailed description.
The authors could perhaps acknowledge the KNMI Climate Explorer (https://climexp.knmi.nl/start.cgi) that also compiled CMIP data and provided them in a standard structure that more easily enabled the analysis of large multi-model ensembles. It reduced the technical barriers to using large numbers of global climate model (GCM) runs.
And perhaps this discussion could include some reference to analysis of large multi-model ensembles as a collective entity, as opposed to comparing individual models (DOI:10.1016/j.cliser.2017.06.013). I’m not sure if people see this subtle difference, and most evaluations involve comparing individual model runs as opposed to how the combined statistics of multi-model ensembles compare with the real world (e.g. standard statistical tests of distributions against observations). There have been some examples where entire CMIP3 (DOI:10.1175/2010JCLI3687.1), CMIP5 (DOI:10.1088/1748-9326/11/5/054017 & DOI: 10.5194/essd-9-905-2017) and CMIP6 (DOI:10.5194/hess-29-45-2025) multi-model ensembles have been downscaled through empirical-statistical downscaling (ESD) which also provide a means for evaluating them through the calibration and diagnostics of the statistical models. Such large ensembles may furthermore capture stochastic regional decadal variability (Deser et al., 2012; 2020; DOI:10.1038/nclimate1562/10.1038/s41558-020-0731-2), and are needed for robust assessments for both impact studies and climate change adaptation.
There are recent evaluations of CMIP GCMs beyond those cited by the authors, e.g. in terms of their ability to reproduce the spatio-temporal covariance structure (DOI:10.5194/gmd-16-2899-2023) and important aspects of the global hydrological cycle (10.1038/s41612-024-00794-z). These contributions may perhaps be relevant for the discussion here. Also the R-shiny app ‘GCMeval’ (DOI:10.1016/j.cliser.2020.100167) was developed for an interactive evaluation of CMIP GCMs, and may merit a mention.
The academic literature citations often miss out important studies which go under the radar and exaggerate the importance of papers written by ‘popular’ scholars. There is a tendency that friends cite each other more and miss out relevant contributions from the wider community, which may be a natural consequence because they talk to each other and learn about each other's work. It’s important to refer to relevant work, even from a wider circle, to ensure highest quality and to acknowledge the work done by colleagues regardless whether they belong to the inner circle or the wider community. It’s hard to keep abreast with all publications, and it may even be a problem with such a large output in terms of publications, but perhaps AI and search tools may help in identifying those that ought to be considered in the literary review/discussion.
Citation: https://doi.org/10.5194/egusphere-2024-3729-CC3 -
CC4: 'Comment on egusphere-2024-3729', Cath Senior, 07 Feb 2025
reply
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2025/egusphere-2024-3729/egusphere-2024-3729-CC4-supplement.pdf
-
RC1: 'Comment on egusphere-2024-3729', Cath Senior, 10 Feb 2025
reply
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2025/egusphere-2024-3729/egusphere-2024-3729-RC1-supplement.pdf
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 545 | 79 | 10 | 634 | 4 | 3 |
- HTML: 545
- PDF: 79
- XML: 10
- Total: 634
- BibTeX: 4
- EndNote: 3
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1