the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 12 Mar 2026
Code accessibility and code quality across phases of the models of the Coupled Model Intercomparison Project
Abstract. This study extends previous research on CMIP5 models to investigate the reproducibility of climate models within the Coupled Model Intercomparison Project (CMIP). It evaluates the accessibility to the source code of the CMIP models through all their phases, emphasizing the need for public repositories to ensure transparency regarding model input, output, and usage rights, along with an analysis of licenses for compliance with scientific standards. A central focus of the research is the assessment of code quality against best practices. In addition, the study examines the historical evolution of computational and code quality across various phases of CMIP, highlighting progress and improving traceability to support scientific reproducibility. We provide valuable insights for future research, proposing solutions and tools designed to improve replicability and enhance project lifecycles that are applicable not only to CMIP but also to broader scientific contexts.
Competing interests: Juan A. Añel, co-author of this paper, is Executive Editor of the journal.
Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors. Views expressed in the text are those of the authors and do not necessarily reflect the views of the publisher.- Preprint
(1666 KB) - Metadata XML
- BibTeX
- EndNote
Status: open (until 28 May 2026)
-
RC1: 'Comment on egusphere-2025-6108', Anonymous Referee #1, 24 Apr 2026
reply
-
AC1: 'Reply on RC1', Michael García Rodríguez, 04 May 2026
reply
We would like to thank the reviewer for the careful reading of the manuscript and for the constructive comments. We greatly appreciate the positive assessment of the relevance of this work for understanding the accessibility, reproducibility, and software practices of CMIP climate models. Below we respond to each point in detail.
Comment 1
We thank the reviewer for this comment and agree that further clarification is useful.
A detailed and comprehensive description of the metrics, methodology, and scoring system implemented in FortranAnalyser is provided in a dedicated article published in Software Impacts (García‑Rodríguez et al., 2024). In that work, the reader can find the full technical explanation of what FortranAnalyser evaluates, including the complete set of static metrics, how they are computed, and how they are aggregated into a final normalized score ranging from 0 to 10. For this reason, the present manuscript includes only a high-level description and cites that companion paper for methodological details.With regard to the interpretation of the score, we agree that the expression “relatively high” requires additional context. This wording is not meant to imply a high absolute level of software quality in a general software engineering sense. Rather, it is used strictly in a comparative, empirical sense, relative to the distribution of scores obtained for the set of accessible CMIP models analyzed in this study.
Specifically, although the theoretical scale of FortranAnalyser spans from 0 to 10, the scores obtained for CMIP models fall within a much narrower empirical range between 2.7 and 4.3, despite covering several CMIP phases. Within this observed range, a value of 4.014 lies in the upper part of the distribution and is therefore relatively high with respect to other operational CMIP models, including many from more recent phases. At the same time, the fact that no model approaches even half of the theoretical maximum score underlines one of the central conclusions of the study: namely, that there remains substantial room for improvement in the structural quality and maintainability of climate model code, even among the best-performing cases.
We will revise the manuscript to better emphasize this relative interpretation and to explicitly clarify that the FortranAnalyser score should not be understood as an absolute measure of software excellence.Comment 2
We thank the reviewer for pointing this out. The overlap in the annexed table is a formatting issue introduced during the compilation of the manuscript. We will carefully revise the layout and ensure that all tables are correctly rendered and fully legible in the revised version.Comment 3
We appreciate the reviewer’s attention to this detail. We will revise the references section and ensure that all “last accessed” notes are consistently written in English, in accordance with the journal’s language standards.Once again, we thank the reviewer for the constructive feedback, which will help us improve the clarity and presentation of the manuscript.
Citation: https://doi.org/10.5194/egusphere-2025-6108-AC1
-
AC1: 'Reply on RC1', Michael García Rodríguez, 04 May 2026
reply
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 578 | 227 | 58 | 863 | 50 | 60 |
- HTML: 578
- PDF: 227
- XML: 58
- Total: 863
- BibTeX: 50
- EndNote: 60
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
In this paper, the authors study the accessibility, understandability, and licensing of climate models used across all CMIP exercises. They found that 18 models out of 121 were accessible, with one being from CMIP3, ten from CMIP5, and seven from CMIP6. Then, with the accessible models, the authors performed code analysis. They found out that all but one were written in Fortran, the exception being NICAM.09. Finally, for the models written in Fortran, they scored the code using the FortranAnalyser, a static code analysis tool that grades Fortran code from zero to ten. Their findings show that the quality of the code has increased with the newer phases of CMIP.
This work is of great importance as it provides an overview of the accessibility of the climate models used in CMIP. Equally important is its discussion regarding the reproducibility of these experiments, which are extremely consequential for advancing our understanding of climate.
I have three critiques for the authors.
1) In section 5, "Code Analysis," I would like to have a clarification, in general terms, of what the FortranAnalyser analyzes to score the models. Moreover, if possible, give an intuition of what a high score code would be. This is because the authors claim that a score of 4.014 is "relatively high," but I have no reference to compare this value with.
2) In the annexed table, where all the models are present, there is overlapped text (page 18, page 20).
3) In the references, the last accessed text is written in Spanish.