the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 02 Jun 2025
Best practices in software development for robust and reproducible geoscientific models based on insights from the Global Carbon Project models
Abstract. Computational models play an increasingly vital role in scientific research, by numerically simulating processes that cannot be solved analytically. Such models are fundamental in geosciences and offer critical insights into the impacts of global change on the Earth system today and in the future. Beyond their value as research tools, models are also software products and should therefore adhere to certain established software engineering standards. However, scientists are rarely trained as software developers, which can lead to potential deficiencies in software quality like unreadable, inefficient, or erroneous code. The complexity of these models, coupled with their integration into broader workflows, also often makes reproducing results, evaluating processes, and building upon them highly challenging.
In this paper, we review the current practices within the development processes of the state-of-the-art land surface models used by the Global Carbon Project. By combining the experience of modelers from the respective research groups with the expertise of professional software engineers, we bridge the gap between software development and scientific modeling to outline key principles and tools for improving software quality in research. We explore four main areas: 1) model testing and validation, 2) scientific, technical, and user documentation, 3) version control, continuous integration, and code review, and 4) the portability and reproducibility of workflows.
Our review of current models reveals that while modeling communities are incorporating many of the suggested practices, significant room for improvement remains in areas such as automated testing, documentation, and reproducible workflows. For instance, there is limited adoption of automated documentation and testing, and provision of reproducible workflow pipelines remains an exception. This highlights the need to identify and promote essential software engineering practices within the scientific community. Nonetheless, we also discuss numerous examples of practices within the community that can serve as guidelines for other models and could even help streamline processes within the entire community.
We conclude with an open-source example implementation of these principles built around the LPJ-GUESS model, showcasing portable and reproducible data flows, a continuous integration setup, and web-based visualizations. This example may serve as a practical resource for model developers, users, and all scientists engaged in scientific programming.
Competing interests: At least one of the (co-)authors is a member of the editorial board of Geoscientific Model Development. The peer-review process was guided by an independent editor, and the authors also have no other competing interests to declare.
Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors. Views expressed in the text are those of the authors and do not necessarily reflect the views of the publisher.-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(1068 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(1068 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2025-1733', Anonymous Referee #1, 23 Aug 2025
Publisher’s note: this comment was edited on 9 September 2025. The following text is not identical to the original comment, but the adjustments were minor without effect on the scientific meaning.
First off all, I want to congratulate the authors on this great piece of work. The manuscript provides really valuable and timely insights, especially now when the scientific community is paying more attention to software quality and making sure geoscientific models can be reliably reproduced. Combining the hands-on experience of modelers within the Global Carbon Project with the perspective of software engineers creates a solid and much-needed approach. Their thorough review of current practices and suggestions for best practices (e.g. covering testing, documentation, continuous integration, portability, and reproducibility) are a big step forward for the carbon modeling community and, more generally, for climate modeling as a whole.
The effort they put into gathering real-world experiences, pinpointing issues, and sharing concrete examples—like the case of LPJ-GUESS—really shows strong teamwork and a real commitment to making scientific software more strong and sustainable. That said, I have some questions and thoughts that popped up while reading this article, and I think they might help deepen the discussion. For example, the article emphasizes that automated testing setups and strategies like CI/CD aren’t widely used, even though they’re really important. It also points out that many models run on high-performance computing (HPC) system which can be expensive and take weeks to run. Given this, what ideas do you have for expanding testing and continuous integration methods to HPC environments and intercomparison projects like CMIP6/CMIP7, where resources are tight and running models costs a lot?
The article also mentions that reproducibility is often challenged by the diversity in environments, things like different compilers, HPC systems, and local setups. Even following FAIR data principles and sharing in open repositories doesn’t always ensure workflows can be repeated easily. Have you thought about ways to tackle these reproducibility barriers across different institutions? Do you see moving towards more standardized containers and environment managers (like Conda, Docker, Singularity) as a practical step to narrowing this gap?
Also, the coexistence of various programming languages such as Fortran, C, and C++, makes it necessary to keep code readable, portable, and maintainable. With the ongoing shift towards newer paradigms, like migrating from Fortran to C++/Python and the increasing use of GPUs and hybrid architectures, what practices would you recommend to keep code sustainable over the next 10–20 years?
I also believe making public code repositories accessible is essential, as platforms like Zenodo (mentioned in the article for sharing model versions) are really effective for preserving and sharing software. Do you think these repositories should become a standard part of our community’s toolkit?
Improve the maintainability and readability of Fortran code in these models is a very hard challenge. Since 13 out of 20 GCP land surface models (LSMs) are written in Fortran, have you considered integrating a specific analyser tool into the development and CI pipelines? Do you think using FortranAnalyser could become a standard for the community, especially since it not only helps find potential errors but also encourages standard coding styles, modularity, and best practices in long-term projects like the GCP LSMs? This open-source tool stands out because it can analyse any version of Fortran, unlike others like F-Lint or fortran-src. This flexibility is especially useful since different Fortran standards coexist in scientific work, and it can even be customized to add specific metrics useful to the development team. If yes, do you see any technical or cultural challenges in adopting it widely?
Citation: https://doi.org/10.5194/egusphere-2025-1733-RC1 -
RC2: 'Reply on RC1', Anonymous Referee #1, 23 Aug 2025
Publisher’s note: the content of this comment was removed on 9 September 2025 since the comment was posted by mistake.
Citation: https://doi.org/10.5194/egusphere-2025-1733-RC2 -
AC3: 'Reply on RC2', Konstantin Gregor, 22 Jan 2026
I am providing a dummy reply here, this was not a real reviewer comment. Please see the other actual reviewer comments to which we replied to.
Citation: https://doi.org/10.5194/egusphere-2025-1733-AC3
-
AC3: 'Reply on RC2', Konstantin Gregor, 22 Jan 2026
-
RC3: 'Reply on RC1', Anonymous Referee #1, 23 Aug 2025
Publisher’s note: the content of this comment was removed on 9 September 2025 since the comment was posted by mistake.
Citation: https://doi.org/10.5194/egusphere-2025-1733-RC3 -
AC4: 'Reply on RC3', Konstantin Gregor, 22 Jan 2026
I am providing a dummy reply here, this was not a real reviewer comment. Please see the other actual reviewer comments to which we replied to.
Citation: https://doi.org/10.5194/egusphere-2025-1733-AC4
-
AC4: 'Reply on RC3', Konstantin Gregor, 22 Jan 2026
-
AC1: 'Reply on RC1', Konstantin Gregor, 20 Jan 2026
Our answers to the reviewers comments are written below, between the original reviewer comments. Our answers are in italic and manuscript parts in bold italic
---
1.
First off all, I want to congratulate the authors on this great piece of work. The manuscript provides really valuable and timely insights, especially now when the scientific community is paying more attention to software quality and making sure geoscientific models can be reliably reproduced. Combining the hands-on experience of modelers within the Global Carbon Project with the perspective of software engineers creates a solid and much-needed approach. Their thorough review of current practices and suggestions for best practices (e.g. covering testing, documentation, continuous integration, portability, and reproducibility) are a big step forward for the carbon modeling community and, more generally, for climate modeling as a whole.
The effort they put into gathering real-world experiences, pinpointing issues, and sharing concrete examples—like the case of LPJ-GUESS—really shows strong teamwork and a real commitment to making scientific software more strong and sustainable. That said, I have some questions and thoughts that popped up while reading this article, and I think they might help deepen the discussion.
We thank the reviewer for these encouraging remarks. We are indeed hopeful that our paper will be helpful for the community.
2.
For example, the article emphasizes that automated testing setups and strategies like CI/CD aren’t widely used, even though they’re really important. It also points out that many models run on high-performance computing (HPC) system which can be expensive and take weeks to run. Given this, what ideas do you have for expanding testing and continuous integration methods to HPC environments and intercomparison projects like CMIP6/CMIP7, where resources are tight and running models costs a lot?
Thank you for this valid remark. We added some information on this issue, highlighting how to efficiently use testing.
In the testing section:
Notably, a test coverage summary shows which processes are not being executed by tests at all and may hint at how the testing setup can be extended to also cover these parts of the model.
In the CI section:
Regarding testing of such complex software pieces like geo-scientific models, the test design plays a key role. We recommend small-scale runs that can already identify issues arising in new implementations without having to run entire global simulations. Reducing the amount of grid cells, or patches, or for instance PFTs can already greatly reduce the computational load while still allowing to test a lot of functionality. Within the community, a common approach is also running global tests for short timescales of a few simulation years, but trying to test all processes of the models. As noted before, test coverage tools may help to pinpoint sections of the code that are not well tested. For larger testing, hybrid parallelization (via the combination of MPI and OpenMP) can reduce parallelization costs, while dynamic load balancing can improve the efficiency of processor work loads. Large-scale tests could run on timed bases, like once per week, or simply be triggered on commit or manually.
3.
The article also mentions that reproducibility is often challenged by the diversity in environments, things like different compilers, HPC systems, and local setups. Even following FAIR data principles and sharing in open repositories doesn’t always ensure workflows can be repeated easily. Have you thought about ways to tackle these reproducibility barriers across different institutions? Do you see moving towards more standardized containers and environment managers (like Conda, Docker, Singularity) as a practical step to narrowing this gap?
This is a very important point. We clarified that these tools are discussed in the manuscript by adding a sentence to the introductory paragraph of section 6 Reproducibility, usability, and portability of scientific workflows and results:
We will discuss how to make workflows reproducible with workflow automation tools like Snakemake and targets, ensuring correct environment setups with package managers like Conda or Renv, and facilitating portability across systems by using container frameworks like Docker or Singularity. Furthermore, we will introduce PEcAn, a tool that can also help make model usage more accessible and repeatable in general.
4.
Also, the coexistence of various programming languages such as Fortran, C, and C++, makes it necessary to keep code readable, portable, and maintainable. With the ongoing shift towards newer paradigms, like migrating from Fortran to C++/Python and the increasing use of GPUs and hybrid architectures, what practices would you recommend to keep code sustainable over the next 10–20 years?
Thank you for this valid remark. We added a short subsection on this topic:
6.2.3 Facilitating portability to new infrastructures and paradigms
The software world is evolving rapidly, with new programming languages, paradigms, environments, and infrastructures emerging continuously. Though at a slower pace, geo-scientific models are also required to be ported to new realms. The practices discussed previously can help address this. In particular, comprehensive test coverage (Sec. 3.1) enables safe refactoring and facilitates porting code to newer versions. Environment managers (Sec. 6.2.1) support updating or replacing dependencies, while containerization (Sec. 6.2.2) helps use software in different infrastructures. In addition, numerous AI-based tools are now available to assist with porting code across languages and platforms. Here, a robust testing framework is also crucial for ensuring correctness throughout such transitions.
5.
I also believe making public code repositories accessible is essential, as platforms like Zenodo (mentioned in the article for sharing model versions) are really effective for preserving and sharing software. Do you think these repositories should become a standard part of our community’s toolkit?
Yes. We added this point in the text:
We propose that using such open repositories should be the standard for all modeling communities. Notably, public version control offers many benefits for the community, as new implementations become readily available and the community can provide feedback and directly contribute to development, testing and assessment of new features. Repositories like Zenodo, on the other hand, are beneficial, because they store data and code versions in a permanent fashion (whereas a group could switch version control systems and historic versions could be lost).
6.
Improve the maintainability and readability of Fortran code in these models is a very hard challenge. Since 13 out of 20 GCP land surface models (LSMs) are written in Fortran, have you considered integrating a specific analyser tool into the development and CI pipelines? Do you think using FortranAnalyser could become a standard for the community, especially since it not only helps find potential errors but also encourages standard coding styles, modularity, and best practices in long-term projects like the GCP LSMs? This open-source tool stands out because it can analyse any version of Fortran, unlike others like F-Lint or fortran-src. This flexibility is especially useful since different Fortran standards coexist in scientific work, and it can even be customized to add specific metrics useful to the development team. If yes, do you see any technical or cultural challenges in adopting it widely?
Yes, this is an important point. We clarified how FortranAnalyser and similar tools can be used in the CI pipeline by restructuring and rewording our section. We refrained from mentioning clear version numbers as the FortranAnalyser website does not specify that it can handle all Fortran versions, and fortran-src also can analyse multiple versions.
The text that was reworded and enhanced is this:
The most commonly used tools for the LSM-languages are clang-tidy for C++, and F-Lint for Fortran. The latter, however, is not freely available. Free alternatives for static Fortran code analysis are, e.g., fortran-src (Contrastin et al., 2025) and FortranAnalyser (García-Rodríguez et al., 2024). Such static code analyses and code formatting should be enforced by integrating the tools into CI pipelines and local development, such as pre-commit hooks, which automatically run them before commits.
Citation: https://doi.org/10.5194/egusphere-2025-1733-AC1 -
AC5: 'Short addition regarding the revised manuscript', Konstantin Gregor, 25 Feb 2026
A short notice: In the process of gathering the information about lines of code and number of developers, we realized that putting it in the table would not be the best option because:
1) LOCs were not really comparable due to some models being part of large ESMs and therefore have a huge number of boilerplate LOC that has nothing to do with the land surface model itself.
2) Number of developers heavily depended also on the larger model system (i.e., is the model part of a climate model / ESM)
3) We did not want to make it seem like some models are better than others because they have higher number of devs or LOC
Therefore, we decided to summarize that information in text and in a pie chart.
Citation: https://doi.org/10.5194/egusphere-2025-1733-AC5
-
AC5: 'Short addition regarding the revised manuscript', Konstantin Gregor, 25 Feb 2026
-
RC2: 'Reply on RC1', Anonymous Referee #1, 23 Aug 2025
-
RC4: 'Comment on egusphere-2025-1733', Anonymous Referee #2, 23 Dec 2025
In its own words, this article aims to combine the experience of land surface modelers and the expertise of professional software engineers in order to define the key principles and tools for improving software quality in the field of research. The authors conclude that it is possible to improve land surface models in areas such as automated testing, documentation, and reproducible workflows, but that inspiration can already be found in individual models, particularly the LPJ-GUESS model, which the authors consider the most successful in this field. This article is well written (with the help of ChatGPT, as explained in the acknowledgements – a honest and fair statement) and can certainly serve as a useful reference, but it can still be greatly improved. I actually tend to challenge the fact that “All [32] authors contributed in writing and editing the manuscript”, or at least that many of them did it more than ChatGPT. This statement may seem provocative, but it stems from a few surprises:
- Why would land surface modelers claim that their models are “the Global Carbon Project models” (title)? Have they not read Table 4 of Friedingstein et al. (2023) or other papers in the series that many of them coauthored? This table lists many more types of models in GCP’s Global Carbon Budget (GCB).
- Why would land surface modelers explain that all models in geoscience including their own ones represent “processes that cannot be solved analytically” (l. 2)?
- Why would land surface modelers suggest (l. 2) that their models are only used to study global change?
- Why would land surface modelers from the GCB ignore the fact that some of them contribute to GCP’s Global Methane Budget (GMB) and to GCP’s Global Nitrous Oxide budget, and that other biogeochemical models contribute to the GMB as well (l. 8-9, 46-51)?
- Why would the modelers who, arguably, wrote Section A 1.7, have the left five lines in German in their section if they have properly read the paper?
- Why would they have left the table (1) describing their model with so little useful information? Rough number of code lines, rough code age, rough developer number (cf. l. 84), number of dedicated scientific programmers (cf. l. 421-422), rough number of systems on which the code has been ported, main configuration (site scale, regional or global scale, coupling with an Atmospheric General Circulation Model, usage as a component of an Earth System Model), and whether they are used in an operational framework or not, would also be interesting to the reader in the paper context (cf. l. 423-424).
- Why would the proud representatives of the 20 land surface models choose to elect one of them as the ideal model (l. 20, l. 68-69, or Section 7 pompously called “A showcase”) without much justification? For example, Section A1.7, with its comprehensive suite of tests run daily, strikes me as impressive (or is the paragraph fake?): it is necessary to discuss the reasons why this model workflow is inferior to the one that was chosen.
- In general, why would they have left their experience in software engineering described in such a superficial way, even suggesting that none of them are professional software engineers (l. 10) ? Exploring the references and URL given Table 1, I see that some of these models have an history of integrating developments from an heterogeneous ecosystem of contributors over several decades (cf. l. 84-85), that several models have been developed under the coordination of weather centers and that some of them have been components of Earth system models of Coupled Model Intercomparison Project (CMIP): some of their developers must be particularly good at software development or the models would not have survived the diversity of their contributors and of their computing environments, and would not have had such challenging applications like CMIP. Actually, the text suggests (l. 4 and 31) that only scientists develop these models. I do believe that some professional software engineers should be credited as well and that some of the scientists involved, formally trained or not (l. 31-32), are also remarkable software developers (see also l. 421-422). The fact that they generalize the dull description of their experience in software engineering to “all sorts of scientific modeling and software” (l. 62-63) may be seen as arrogant. Developers of Numerical Weather or Ocean Prediction, for instance, would be mere amateurs? Come on! They deserve better comments, and the LSM models as well.
- Why would land surface modelers who are familiar with MIPs explain that MIPs allow them to “understand the range of uncertainty” (l. 227 – what does it mean?), while also forgetting to mention that a main outcome of MIPs is debugging the weakest models? A deeper investigation of MIP usefulness is certainly not out of the scope of this paper (l. 231).
I really urge the 32 coauthors to deepen their analysis for the benefit of the readers. In reality, the LSM authors could have presented their software work in a much more positive light and highlighted the merit and strength of their efforts, whereas the article reads like a simple request for funding. Note that in their work, LSM developers also face the bugs of commercial software, like compilers: nobody’s safe.
Additional detailed comments:
- It would be worth explaining that the entry ticket for LSMs in the GCB is rather cheap, as explained in Section S.4.2 of Friedlingstein et al. (2023): “We apply three criteria for minimum DGVMs realism by including only those DGVMs with (1) steady state after spin up, (2) global net land flux (SLAND – ELUC) that is an atmosphere-to-land carbon flux over the 1990s ranging between -0.3 and 2.3 GtC yr-1, within 90% confidence of constraints by global atmospheric and oceanic observations (Keeling and Manning, 2014; Wanninkhof et al., 2013), and (3) global ELUC that is a carbon source to the atmosphere over the 1990s, as already mentioned in Supplement S.2.2. All DGVMs meet these three criteria.” As a consequence, the quality of the selected LSMs is likely heterogeneous and their engineering support as well.
- Surprisingly, the issue of competition with the private sector for the recruitment of skilled computer engineers is not addressed at all.
- The challenge of rewriting hundreds of thousands of lines of code and the dilemma of scientists losing their understanding of the rewritten versions must be addressed.
- Line 2: why is the discussion restricted to the impact of global change?
- Line 49: what are the critical insights offered by the GCB for policy-making?
- Line 50: where is the land flux to the ocean computed by LSMs used in the GCB?
- Line 33: the statement about the lack of recognition for software development in academia is unfair, in particular if one relates it to l. 255-256 about dedicated high-profile scientific journals for software development. There are also many calls for proposal in some countries or groups of countries for funding software developments or just to offer professional software support.
- Line 86 and 631-632: the lack of funding for positions dedicated to scientific programming is also a choice made by scientists who set priorities within their host organizations.
- Line 91: around me, the time spent on software development or on software support calls by scientists is a choice. It may not be sufficient, but authors should not blame the system in the first place, but rather themselves.
- Line 124-125: to detect where something is wrong, reference outputs need to be available, which is not always the case. Think about the adjoint code of a LSM routine for instance (except if one builds a heavy adjoint testing machinery around each routine).
- Line 219: netCDF has already been used above without any expansion of the acronym.
- Line 224: MIP has not been defined.
- Line 455: missing punctuation mark.
- Appendix A: the authors should try to better exploit/integrate this appendix within the main text.
- 6: this subsection seems to be out of scope. It should be either removed or rewritten in order to fit the paper.
- In Table 1, the citation Vuichard et al has been duplicated and Hoffman et al (l. 213) has no year associated.
Citation: https://doi.org/10.5194/egusphere-2025-1733-RC4 -
AC2: 'Reply on RC4', Konstantin Gregor, 20 Jan 2026
We would like to thank the reviewer for their thorough assessment of our manuscript, which pointed out inconsistencies and flaws. We addressed these aspects and believe this significantly improved our manuscript.
We would like to respond to the four major criticisms, namely 1) the use of LLMs for text editing, 2) the usage of one specific model for the example, 3) the criticism of the capability of scientists in terms of programming, and 4) the situation of funding.
The comments by the reviewer were quite critical but raised important points, on most of which the authors agree. We are confident that addressing them has improved the paper by quite a lot. In the following we will address the comments in more detail:First, we have to refute the claim about the author’s contributions and AI use. Indeed, all authors contributed intellectually to different parts of the manuscript. The fact that still inconsistencies and flaws were found shows the importance of an external and unbiased peer review. We did use ChatGPT for language editing, but we have hundreds of emails, 20+ survey responses, and intermediate paper versions from well over one year to challenge that this paper or even parts of it were AI-generated. In fact, this paper started as a short paper by the first author on coding tips for modeling communities. An initial submission was challenged by the editors of GMD because it was not detailed enough, and it was suggested to include insights from other model communities. This led to a constantly growing manuscript with the insights of now 32 people, and offers a scope much larger than that of recently published papers that focus on single aspects (like Docker, version control, code review, FAIR, etc., see References 1–5 below).
Second, the reviewer wonders why the model LPJ-GUESS was stated as “most successful in the field”. While we neither believe nor say this, we did realize now that we used it for most examples, simply because the first author is most familiar with it. We now removed the mention of this particular model in almost all cases because while the example figures or code snippets are based on this model, they are relevant for all models. In addition to our existing disclaimer about examples not being endorsements/criticism in the introduction, we added more claims like this, and more clearly indicate that for the pipeline example (previously “showcase”) we selected one model for pragmatic reasons and reduced the mentioning of which model we used for this to a minimum. We also worked in more references to the appendix (examples of all the model communities) to make sure benefits of numerous models are highlighted.
Third, the reviewer raises concerns about our criticism of the work/skill of the community. We realize now that the tone we picked was not ideal and more negative than it was intended to be (notably, considering that the author team represents numerous models, we were criticizing our own work). We have thoroughly revised the manuscript to adopt a neutral tone.
Fourth, the reviewer disagrees with our claims about the lack of time and money in this field, because they have enough time and money. For us, this highlights the relevance of including authors working with multiple models. The situation described by the reviewer does not align with the experience raised by multiple co-authors of this paper as well as that of surveys which we cite, therefore we prefer to keep this notion in the paper but extended the claims to show that experiences may differ.
---
Since the formatting options of the reply text box are very limited, and since there were a large number of detailed comments, leading also to complex changes in the manuscript, we decided to provide the answer to the reviewer's comments in an attached PDF file. We believe that this makes it easier to assess the original comments and the answers and corresponding changes.
On behalf of all co-authors,
Konstantin Gregor
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2025-1733', Anonymous Referee #1, 23 Aug 2025
Publisher’s note: this comment was edited on 9 September 2025. The following text is not identical to the original comment, but the adjustments were minor without effect on the scientific meaning.
First off all, I want to congratulate the authors on this great piece of work. The manuscript provides really valuable and timely insights, especially now when the scientific community is paying more attention to software quality and making sure geoscientific models can be reliably reproduced. Combining the hands-on experience of modelers within the Global Carbon Project with the perspective of software engineers creates a solid and much-needed approach. Their thorough review of current practices and suggestions for best practices (e.g. covering testing, documentation, continuous integration, portability, and reproducibility) are a big step forward for the carbon modeling community and, more generally, for climate modeling as a whole.
The effort they put into gathering real-world experiences, pinpointing issues, and sharing concrete examples—like the case of LPJ-GUESS—really shows strong teamwork and a real commitment to making scientific software more strong and sustainable. That said, I have some questions and thoughts that popped up while reading this article, and I think they might help deepen the discussion. For example, the article emphasizes that automated testing setups and strategies like CI/CD aren’t widely used, even though they’re really important. It also points out that many models run on high-performance computing (HPC) system which can be expensive and take weeks to run. Given this, what ideas do you have for expanding testing and continuous integration methods to HPC environments and intercomparison projects like CMIP6/CMIP7, where resources are tight and running models costs a lot?
The article also mentions that reproducibility is often challenged by the diversity in environments, things like different compilers, HPC systems, and local setups. Even following FAIR data principles and sharing in open repositories doesn’t always ensure workflows can be repeated easily. Have you thought about ways to tackle these reproducibility barriers across different institutions? Do you see moving towards more standardized containers and environment managers (like Conda, Docker, Singularity) as a practical step to narrowing this gap?
Also, the coexistence of various programming languages such as Fortran, C, and C++, makes it necessary to keep code readable, portable, and maintainable. With the ongoing shift towards newer paradigms, like migrating from Fortran to C++/Python and the increasing use of GPUs and hybrid architectures, what practices would you recommend to keep code sustainable over the next 10–20 years?
I also believe making public code repositories accessible is essential, as platforms like Zenodo (mentioned in the article for sharing model versions) are really effective for preserving and sharing software. Do you think these repositories should become a standard part of our community’s toolkit?
Improve the maintainability and readability of Fortran code in these models is a very hard challenge. Since 13 out of 20 GCP land surface models (LSMs) are written in Fortran, have you considered integrating a specific analyser tool into the development and CI pipelines? Do you think using FortranAnalyser could become a standard for the community, especially since it not only helps find potential errors but also encourages standard coding styles, modularity, and best practices in long-term projects like the GCP LSMs? This open-source tool stands out because it can analyse any version of Fortran, unlike others like F-Lint or fortran-src. This flexibility is especially useful since different Fortran standards coexist in scientific work, and it can even be customized to add specific metrics useful to the development team. If yes, do you see any technical or cultural challenges in adopting it widely?
Citation: https://doi.org/10.5194/egusphere-2025-1733-RC1 -
RC2: 'Reply on RC1', Anonymous Referee #1, 23 Aug 2025
Publisher’s note: the content of this comment was removed on 9 September 2025 since the comment was posted by mistake.
Citation: https://doi.org/10.5194/egusphere-2025-1733-RC2 -
AC3: 'Reply on RC2', Konstantin Gregor, 22 Jan 2026
I am providing a dummy reply here, this was not a real reviewer comment. Please see the other actual reviewer comments to which we replied to.
Citation: https://doi.org/10.5194/egusphere-2025-1733-AC3
-
AC3: 'Reply on RC2', Konstantin Gregor, 22 Jan 2026
-
RC3: 'Reply on RC1', Anonymous Referee #1, 23 Aug 2025
Publisher’s note: the content of this comment was removed on 9 September 2025 since the comment was posted by mistake.
Citation: https://doi.org/10.5194/egusphere-2025-1733-RC3 -
AC4: 'Reply on RC3', Konstantin Gregor, 22 Jan 2026
I am providing a dummy reply here, this was not a real reviewer comment. Please see the other actual reviewer comments to which we replied to.
Citation: https://doi.org/10.5194/egusphere-2025-1733-AC4
-
AC4: 'Reply on RC3', Konstantin Gregor, 22 Jan 2026
-
AC1: 'Reply on RC1', Konstantin Gregor, 20 Jan 2026
Our answers to the reviewers comments are written below, between the original reviewer comments. Our answers are in italic and manuscript parts in bold italic
---
1.
First off all, I want to congratulate the authors on this great piece of work. The manuscript provides really valuable and timely insights, especially now when the scientific community is paying more attention to software quality and making sure geoscientific models can be reliably reproduced. Combining the hands-on experience of modelers within the Global Carbon Project with the perspective of software engineers creates a solid and much-needed approach. Their thorough review of current practices and suggestions for best practices (e.g. covering testing, documentation, continuous integration, portability, and reproducibility) are a big step forward for the carbon modeling community and, more generally, for climate modeling as a whole.
The effort they put into gathering real-world experiences, pinpointing issues, and sharing concrete examples—like the case of LPJ-GUESS—really shows strong teamwork and a real commitment to making scientific software more strong and sustainable. That said, I have some questions and thoughts that popped up while reading this article, and I think they might help deepen the discussion.
We thank the reviewer for these encouraging remarks. We are indeed hopeful that our paper will be helpful for the community.
2.
For example, the article emphasizes that automated testing setups and strategies like CI/CD aren’t widely used, even though they’re really important. It also points out that many models run on high-performance computing (HPC) system which can be expensive and take weeks to run. Given this, what ideas do you have for expanding testing and continuous integration methods to HPC environments and intercomparison projects like CMIP6/CMIP7, where resources are tight and running models costs a lot?
Thank you for this valid remark. We added some information on this issue, highlighting how to efficiently use testing.
In the testing section:
Notably, a test coverage summary shows which processes are not being executed by tests at all and may hint at how the testing setup can be extended to also cover these parts of the model.
In the CI section:
Regarding testing of such complex software pieces like geo-scientific models, the test design plays a key role. We recommend small-scale runs that can already identify issues arising in new implementations without having to run entire global simulations. Reducing the amount of grid cells, or patches, or for instance PFTs can already greatly reduce the computational load while still allowing to test a lot of functionality. Within the community, a common approach is also running global tests for short timescales of a few simulation years, but trying to test all processes of the models. As noted before, test coverage tools may help to pinpoint sections of the code that are not well tested. For larger testing, hybrid parallelization (via the combination of MPI and OpenMP) can reduce parallelization costs, while dynamic load balancing can improve the efficiency of processor work loads. Large-scale tests could run on timed bases, like once per week, or simply be triggered on commit or manually.
3.
The article also mentions that reproducibility is often challenged by the diversity in environments, things like different compilers, HPC systems, and local setups. Even following FAIR data principles and sharing in open repositories doesn’t always ensure workflows can be repeated easily. Have you thought about ways to tackle these reproducibility barriers across different institutions? Do you see moving towards more standardized containers and environment managers (like Conda, Docker, Singularity) as a practical step to narrowing this gap?
This is a very important point. We clarified that these tools are discussed in the manuscript by adding a sentence to the introductory paragraph of section 6 Reproducibility, usability, and portability of scientific workflows and results:
We will discuss how to make workflows reproducible with workflow automation tools like Snakemake and targets, ensuring correct environment setups with package managers like Conda or Renv, and facilitating portability across systems by using container frameworks like Docker or Singularity. Furthermore, we will introduce PEcAn, a tool that can also help make model usage more accessible and repeatable in general.
4.
Also, the coexistence of various programming languages such as Fortran, C, and C++, makes it necessary to keep code readable, portable, and maintainable. With the ongoing shift towards newer paradigms, like migrating from Fortran to C++/Python and the increasing use of GPUs and hybrid architectures, what practices would you recommend to keep code sustainable over the next 10–20 years?
Thank you for this valid remark. We added a short subsection on this topic:
6.2.3 Facilitating portability to new infrastructures and paradigms
The software world is evolving rapidly, with new programming languages, paradigms, environments, and infrastructures emerging continuously. Though at a slower pace, geo-scientific models are also required to be ported to new realms. The practices discussed previously can help address this. In particular, comprehensive test coverage (Sec. 3.1) enables safe refactoring and facilitates porting code to newer versions. Environment managers (Sec. 6.2.1) support updating or replacing dependencies, while containerization (Sec. 6.2.2) helps use software in different infrastructures. In addition, numerous AI-based tools are now available to assist with porting code across languages and platforms. Here, a robust testing framework is also crucial for ensuring correctness throughout such transitions.
5.
I also believe making public code repositories accessible is essential, as platforms like Zenodo (mentioned in the article for sharing model versions) are really effective for preserving and sharing software. Do you think these repositories should become a standard part of our community’s toolkit?
Yes. We added this point in the text:
We propose that using such open repositories should be the standard for all modeling communities. Notably, public version control offers many benefits for the community, as new implementations become readily available and the community can provide feedback and directly contribute to development, testing and assessment of new features. Repositories like Zenodo, on the other hand, are beneficial, because they store data and code versions in a permanent fashion (whereas a group could switch version control systems and historic versions could be lost).
6.
Improve the maintainability and readability of Fortran code in these models is a very hard challenge. Since 13 out of 20 GCP land surface models (LSMs) are written in Fortran, have you considered integrating a specific analyser tool into the development and CI pipelines? Do you think using FortranAnalyser could become a standard for the community, especially since it not only helps find potential errors but also encourages standard coding styles, modularity, and best practices in long-term projects like the GCP LSMs? This open-source tool stands out because it can analyse any version of Fortran, unlike others like F-Lint or fortran-src. This flexibility is especially useful since different Fortran standards coexist in scientific work, and it can even be customized to add specific metrics useful to the development team. If yes, do you see any technical or cultural challenges in adopting it widely?
Yes, this is an important point. We clarified how FortranAnalyser and similar tools can be used in the CI pipeline by restructuring and rewording our section. We refrained from mentioning clear version numbers as the FortranAnalyser website does not specify that it can handle all Fortran versions, and fortran-src also can analyse multiple versions.
The text that was reworded and enhanced is this:
The most commonly used tools for the LSM-languages are clang-tidy for C++, and F-Lint for Fortran. The latter, however, is not freely available. Free alternatives for static Fortran code analysis are, e.g., fortran-src (Contrastin et al., 2025) and FortranAnalyser (García-Rodríguez et al., 2024). Such static code analyses and code formatting should be enforced by integrating the tools into CI pipelines and local development, such as pre-commit hooks, which automatically run them before commits.
Citation: https://doi.org/10.5194/egusphere-2025-1733-AC1 -
AC5: 'Short addition regarding the revised manuscript', Konstantin Gregor, 25 Feb 2026
A short notice: In the process of gathering the information about lines of code and number of developers, we realized that putting it in the table would not be the best option because:
1) LOCs were not really comparable due to some models being part of large ESMs and therefore have a huge number of boilerplate LOC that has nothing to do with the land surface model itself.
2) Number of developers heavily depended also on the larger model system (i.e., is the model part of a climate model / ESM)
3) We did not want to make it seem like some models are better than others because they have higher number of devs or LOC
Therefore, we decided to summarize that information in text and in a pie chart.
Citation: https://doi.org/10.5194/egusphere-2025-1733-AC5
-
AC5: 'Short addition regarding the revised manuscript', Konstantin Gregor, 25 Feb 2026
-
RC2: 'Reply on RC1', Anonymous Referee #1, 23 Aug 2025
-
RC4: 'Comment on egusphere-2025-1733', Anonymous Referee #2, 23 Dec 2025
In its own words, this article aims to combine the experience of land surface modelers and the expertise of professional software engineers in order to define the key principles and tools for improving software quality in the field of research. The authors conclude that it is possible to improve land surface models in areas such as automated testing, documentation, and reproducible workflows, but that inspiration can already be found in individual models, particularly the LPJ-GUESS model, which the authors consider the most successful in this field. This article is well written (with the help of ChatGPT, as explained in the acknowledgements – a honest and fair statement) and can certainly serve as a useful reference, but it can still be greatly improved. I actually tend to challenge the fact that “All [32] authors contributed in writing and editing the manuscript”, or at least that many of them did it more than ChatGPT. This statement may seem provocative, but it stems from a few surprises:
- Why would land surface modelers claim that their models are “the Global Carbon Project models” (title)? Have they not read Table 4 of Friedingstein et al. (2023) or other papers in the series that many of them coauthored? This table lists many more types of models in GCP’s Global Carbon Budget (GCB).
- Why would land surface modelers explain that all models in geoscience including their own ones represent “processes that cannot be solved analytically” (l. 2)?
- Why would land surface modelers suggest (l. 2) that their models are only used to study global change?
- Why would land surface modelers from the GCB ignore the fact that some of them contribute to GCP’s Global Methane Budget (GMB) and to GCP’s Global Nitrous Oxide budget, and that other biogeochemical models contribute to the GMB as well (l. 8-9, 46-51)?
- Why would the modelers who, arguably, wrote Section A 1.7, have the left five lines in German in their section if they have properly read the paper?
- Why would they have left the table (1) describing their model with so little useful information? Rough number of code lines, rough code age, rough developer number (cf. l. 84), number of dedicated scientific programmers (cf. l. 421-422), rough number of systems on which the code has been ported, main configuration (site scale, regional or global scale, coupling with an Atmospheric General Circulation Model, usage as a component of an Earth System Model), and whether they are used in an operational framework or not, would also be interesting to the reader in the paper context (cf. l. 423-424).
- Why would the proud representatives of the 20 land surface models choose to elect one of them as the ideal model (l. 20, l. 68-69, or Section 7 pompously called “A showcase”) without much justification? For example, Section A1.7, with its comprehensive suite of tests run daily, strikes me as impressive (or is the paragraph fake?): it is necessary to discuss the reasons why this model workflow is inferior to the one that was chosen.
- In general, why would they have left their experience in software engineering described in such a superficial way, even suggesting that none of them are professional software engineers (l. 10) ? Exploring the references and URL given Table 1, I see that some of these models have an history of integrating developments from an heterogeneous ecosystem of contributors over several decades (cf. l. 84-85), that several models have been developed under the coordination of weather centers and that some of them have been components of Earth system models of Coupled Model Intercomparison Project (CMIP): some of their developers must be particularly good at software development or the models would not have survived the diversity of their contributors and of their computing environments, and would not have had such challenging applications like CMIP. Actually, the text suggests (l. 4 and 31) that only scientists develop these models. I do believe that some professional software engineers should be credited as well and that some of the scientists involved, formally trained or not (l. 31-32), are also remarkable software developers (see also l. 421-422). The fact that they generalize the dull description of their experience in software engineering to “all sorts of scientific modeling and software” (l. 62-63) may be seen as arrogant. Developers of Numerical Weather or Ocean Prediction, for instance, would be mere amateurs? Come on! They deserve better comments, and the LSM models as well.
- Why would land surface modelers who are familiar with MIPs explain that MIPs allow them to “understand the range of uncertainty” (l. 227 – what does it mean?), while also forgetting to mention that a main outcome of MIPs is debugging the weakest models? A deeper investigation of MIP usefulness is certainly not out of the scope of this paper (l. 231).
I really urge the 32 coauthors to deepen their analysis for the benefit of the readers. In reality, the LSM authors could have presented their software work in a much more positive light and highlighted the merit and strength of their efforts, whereas the article reads like a simple request for funding. Note that in their work, LSM developers also face the bugs of commercial software, like compilers: nobody’s safe.
Additional detailed comments:
- It would be worth explaining that the entry ticket for LSMs in the GCB is rather cheap, as explained in Section S.4.2 of Friedlingstein et al. (2023): “We apply three criteria for minimum DGVMs realism by including only those DGVMs with (1) steady state after spin up, (2) global net land flux (SLAND – ELUC) that is an atmosphere-to-land carbon flux over the 1990s ranging between -0.3 and 2.3 GtC yr-1, within 90% confidence of constraints by global atmospheric and oceanic observations (Keeling and Manning, 2014; Wanninkhof et al., 2013), and (3) global ELUC that is a carbon source to the atmosphere over the 1990s, as already mentioned in Supplement S.2.2. All DGVMs meet these three criteria.” As a consequence, the quality of the selected LSMs is likely heterogeneous and their engineering support as well.
- Surprisingly, the issue of competition with the private sector for the recruitment of skilled computer engineers is not addressed at all.
- The challenge of rewriting hundreds of thousands of lines of code and the dilemma of scientists losing their understanding of the rewritten versions must be addressed.
- Line 2: why is the discussion restricted to the impact of global change?
- Line 49: what are the critical insights offered by the GCB for policy-making?
- Line 50: where is the land flux to the ocean computed by LSMs used in the GCB?
- Line 33: the statement about the lack of recognition for software development in academia is unfair, in particular if one relates it to l. 255-256 about dedicated high-profile scientific journals for software development. There are also many calls for proposal in some countries or groups of countries for funding software developments or just to offer professional software support.
- Line 86 and 631-632: the lack of funding for positions dedicated to scientific programming is also a choice made by scientists who set priorities within their host organizations.
- Line 91: around me, the time spent on software development or on software support calls by scientists is a choice. It may not be sufficient, but authors should not blame the system in the first place, but rather themselves.
- Line 124-125: to detect where something is wrong, reference outputs need to be available, which is not always the case. Think about the adjoint code of a LSM routine for instance (except if one builds a heavy adjoint testing machinery around each routine).
- Line 219: netCDF has already been used above without any expansion of the acronym.
- Line 224: MIP has not been defined.
- Line 455: missing punctuation mark.
- Appendix A: the authors should try to better exploit/integrate this appendix within the main text.
- 6: this subsection seems to be out of scope. It should be either removed or rewritten in order to fit the paper.
- In Table 1, the citation Vuichard et al has been duplicated and Hoffman et al (l. 213) has no year associated.
Citation: https://doi.org/10.5194/egusphere-2025-1733-RC4 -
AC2: 'Reply on RC4', Konstantin Gregor, 20 Jan 2026
We would like to thank the reviewer for their thorough assessment of our manuscript, which pointed out inconsistencies and flaws. We addressed these aspects and believe this significantly improved our manuscript.
We would like to respond to the four major criticisms, namely 1) the use of LLMs for text editing, 2) the usage of one specific model for the example, 3) the criticism of the capability of scientists in terms of programming, and 4) the situation of funding.
The comments by the reviewer were quite critical but raised important points, on most of which the authors agree. We are confident that addressing them has improved the paper by quite a lot. In the following we will address the comments in more detail:First, we have to refute the claim about the author’s contributions and AI use. Indeed, all authors contributed intellectually to different parts of the manuscript. The fact that still inconsistencies and flaws were found shows the importance of an external and unbiased peer review. We did use ChatGPT for language editing, but we have hundreds of emails, 20+ survey responses, and intermediate paper versions from well over one year to challenge that this paper or even parts of it were AI-generated. In fact, this paper started as a short paper by the first author on coding tips for modeling communities. An initial submission was challenged by the editors of GMD because it was not detailed enough, and it was suggested to include insights from other model communities. This led to a constantly growing manuscript with the insights of now 32 people, and offers a scope much larger than that of recently published papers that focus on single aspects (like Docker, version control, code review, FAIR, etc., see References 1–5 below).
Second, the reviewer wonders why the model LPJ-GUESS was stated as “most successful in the field”. While we neither believe nor say this, we did realize now that we used it for most examples, simply because the first author is most familiar with it. We now removed the mention of this particular model in almost all cases because while the example figures or code snippets are based on this model, they are relevant for all models. In addition to our existing disclaimer about examples not being endorsements/criticism in the introduction, we added more claims like this, and more clearly indicate that for the pipeline example (previously “showcase”) we selected one model for pragmatic reasons and reduced the mentioning of which model we used for this to a minimum. We also worked in more references to the appendix (examples of all the model communities) to make sure benefits of numerous models are highlighted.
Third, the reviewer raises concerns about our criticism of the work/skill of the community. We realize now that the tone we picked was not ideal and more negative than it was intended to be (notably, considering that the author team represents numerous models, we were criticizing our own work). We have thoroughly revised the manuscript to adopt a neutral tone.
Fourth, the reviewer disagrees with our claims about the lack of time and money in this field, because they have enough time and money. For us, this highlights the relevance of including authors working with multiple models. The situation described by the reviewer does not align with the experience raised by multiple co-authors of this paper as well as that of surveys which we cite, therefore we prefer to keep this notion in the paper but extended the claims to show that experiences may differ.
---
Since the formatting options of the reply text box are very limited, and since there were a large number of detailed comments, leading also to complex changes in the manuscript, we decided to provide the answer to the reviewer's comments in an attached PDF file. We believe that this makes it easier to assess the original comments and the answers and corresponding changes.
On behalf of all co-authors,
Konstantin Gregor
Peer review completion
Journal article(s) based on this preprint
Model code and software
Model workflow showcase Konstantin Gregor https://doi.org/10.5281/zenodo.15191116
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 2,685 | 409 | 59 | 3,153 | 56 | 74 |
- HTML: 2,685
- PDF: 409
- XML: 59
- Total: 3,153
- BibTeX: 56
- EndNote: 74
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Konstantin Gregor
Benjamin F. Meyer
Tillmann Gaida
Victor Justo Vasquez
Karina Bett-Williams
Matthew Forrest
João P. Darela-Filho
Sam Rabin
Marcos Longo
Joe R. Melton
Johan Nord
Peter Anthoni
Vladislav Bastrikov
Thomas Colligan
Christine Delire
Michael C. Dietze
George Hurtt
Akihiko Ito
Lasse T. Keetz
Jürgen Knauer
Johannes Köster
Tzu-Shun Lin
Lei Ma
Marie Minvielle
Stefan Olin
Sebastian Ostberg
Reiner Schnur
Urs Schönenberger
Peter E. Thornton
Anja Rammig
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(1068 KB) - Metadata XML