the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Calibrating the GAMIL3-1° climate model using a derivative-free optimization method
Abstract. Parameterization in climate models often involves parameters that are poorly constrained by observations or theoretical understanding alone. Manual tuning by experts can be time-consuming, subjective, and prone to underestimating uncertainties. Automated tuning methods offer a promising alternative, enabling faster, objective improvements in model performance and better uncertainty quantification. This study presents an automated parameter-tuning framework that employs a derivative-free optimization solver (DFO-LS) to simultaneously perturb and tune multiple convection-related and microphysics parameters. The framework explicitly accounts for observational and initial condition uncertainties (internal variability) to calibrate a 1-degree resolution atmospheric model (GAMIL3). Two experiments, adjusting 10 and 20 parameters, were conducted alongside three sensitivity experiments that varied initial parameter values for a 10-parameter case. Both of the first two experiments showed a rapid decrease in the cost function, with the 10-parameter optimization significantly improving model accuracy in 24 out of 34 variables. Expanding to 20 parameters further enhanced accuracy, with improvement in 25 of 34 variables, though some structural model errors emerged. Ten-year AMIP simulations validated the robustness and stability of the tuning results, showing that the improvements persisted over extended simulations. Additionally, evaluations of the coupled model with optimized parameters showed–compare to the default parameter setting–reduced climate drift, a more stable climate system, and more realistic sea surface temperatures, despite a slight energy imbalance and some regional biases. The sensitivity experiments underscored the efficiency of the tuning algorithm and highlight the importance of expert judgment in selecting initial parameter values. This tuning framework is broadly applicable to other general circulation models (GCMs), supporting comprehensive parameter tuning and advancing model development.
- Preprint
(3612 KB) - Metadata XML
-
Supplement
(763 KB) - BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on egusphere-2024-3770', Anonymous Referee #1, 21 Apr 2025
This well-structured manuscript presents a novel approach for climate model-tuning and the results that such tuning yields for a given model (GAMIL3) under 3 different model configurations: 1 year AMIP for tuning , 10 year AMIP and 30 year coupled pre-industrial Control. The presented tuning method is potentially relevant for other climate models. The authors show that the DFO-LS method is able to systematically improve the ‘a priori’ model parameter values and that the improvements hold across the different model configurations. The text is well written, with some potential however for more precise and less verbose language. In general, the manuscript could improve by adding some comparison or references to similar past efforts on model tuning, but I acknowledge that often findings and results are quite model-specific.
Specific comments
L45-46 Some references would be welcome.
L186: Not strictly necessary, but perhaps having a sketch showing the sequence of experiments performed would help the reader.
L186: The text has no literature reference for GAMIL3. If no documentation exists for this model version, a more detailed description of it would be needed, as an Appendix if needed. The current description between L187-203 is vague and full of ambiguities (‘updates to the planetary boundary layer scheme’, ’GAMIL3 integrates several parametrizations recommended by CMIP6’)
L280: Is there any reason or reference why you would give twice as much weight to C_i than to C_0?
L296: put the definition of the Jacobians in context. Why are you presenting it? where in the paper is used?
L351: Why ke and captlmt are explicity mentioned? please explain
L414: Any illustrative example of compensating errors in the model?
L503: I’d re-name this section as ‘’Coupled model evaluation’
L521: lower rhcrit could, a priori, also enhance precip. Lower rhcrit would enhance convection and, this, precipitation. Even if it is not the case in the simulations, it may be worth being mentioned.
L533: contributing to the decreased low-level cloud fraction and further reducing precipitation (since this was mentioned in the previous paragraph)
L569: Describe for how long the coupled model was run, one can only infer it from the Figures.
L569: for coupled simulations it is quite relevant to explain how the land, and specially the ocean, were initialized. This is relevant because a perfect model should drift if the ocean is not correctly initialized, and you would not like to tune your model to compensate for an ocean-caused drift.
L575: While the reduction of OLR is obvious (and interrelated) to the drop of T2M, the reduction in RSR seems to have a more complex mechanism and would merit an additional explanatory sentence.
L718: a leak of 1.4 W/mw seems quite relevant to me, and , besides being present here, it should have been mentioned earlier in the results when discussing NETFLUX.
L727: Mention that the primary experiments where 1 -year long AMIP
L740: the maintained improvement over extended periods is good news given that you tuned on a single year and ignored interannual variability. Could you hypothesise whether (and how much) you would expect a better tuning if you optimize the parameters over several years of AMIP?technical corrections
L51 difficult to understand the complete sentence. Perhaps ‘carbon cycle or nutrient cycles’ would clarify it.
L60: remove ‘computational constrains’ as it only adds confusion to the sentence
L239: ‘discussed in a later section’. Please state at which specific section.
L250: listed in the first [instead of last] column of Table 2
L254: listed in the first [instead of last] column of Table 2
L273-L277: Break the sentence, it is difficult to follow. L288-L291: assuming there are no typos in the equations, there is inconsistent information in these lines: N is defined twice and differently, and C is defined although missing in the equation.
L357: why not just mention total number of iterations, instead of excluding the first 10?
L403: remove ‘’an
L464: variables
L475: this is less succesfull, in relative terms, than the 10 parameter case.
L486: exhibit similar behaviour L603: which improvements for which case?
L606: flux of energy towards the ocean, instead of ocean surface flux.
L691: a common issue
All figures: larger legends would be good.
Table 2: add units (if they have) to the parameters, as it may help to understand their role.
Figure 2: the numbers written in the experiment color code are very hard to read. Also, the caption does not explain what they mean, nor the meaning of the vertical dashed lines in b) and c).
Figure 3: I would rename AMIP@10years by AMIP2005-2014, here and wherever mentioned in the text.
Figure 7: there is a red ‘v’
Figure 8: percent instead of precent
Figure 12: change colorcode as it uses the same as Figure 7. In Fig 7, however, the numbers in the Table display the actual Jacobians, while here it displays the range between Jacobians. A change of colorcode would help explain that we are not looking at the exact same metric.Citation: https://doi.org/10.5194/egusphere-2024-3770-RC1 -
RC2: 'Comment on egusphere-2024-3770', Anonymous Referee #2, 31 May 2025
This study presents a derivative-free optimization framework for tuning climate model parameters. The framework was applied to the GAMIL3 atmospheric model and evaluated for both 10-parameter and 20-parameter cases. The study assessed the framework's effectiveness in terms of the initial selection of model parameter values and found that the initial selection of model parameter values considerably affects the tuning results. The study also evaluated the effectiveness of applying the optimized model parameters, derived from the atmospheric model, to an atmosphere-ocean coupled climate model. Model parameterization optimization and model tuning are important aspects in the climate modeling community. The paper is well written and worth publishing. However, to benefit a wider modeling community, some issues need to be addressed and further clarification is necessary.
L174-175: Please provide more details about the initial trust region and parameter constraints. Is there any difference between parameter constraints and parameters' plausible ranges?
L180: In each iteration of the optimization process, how many simulations are conducted?
L215: A 30-year simulation is insufficient to fully evaluate the effectiveness of the modified model parameters in a fully coupled model.
L226-228: \theta is not defined.
L230-231: _TROPICALLAND, _TROPICALOCEAN, _NHX and _SHX are not defined.
L236: LAT is not defined.
L237-238: Please clarify how the uncertainty is derived from the absolute error.
L250: I can’t find them in the last column of Table 2.
L405-407: The tuning process of the 20-parameter case was affected by using the same initial perturbations for the original 10 parameters. It is important to evaluate the effectiveness of the tuning method in terms of adding more parameters by comparing the 10-parameter and 20-parameter cases with independent initial parameter perturbations.
L416-417: What does “the initial 20 runs” refer to? Are these the initial perturbation runs conducted before the optimizing iterations begin? If so, please clarify this point. It appears that both the 10-parameter and 20-parameter cases achieve nearly the same STABLE performance by the 21 iterations. Does this mean the total number of runs for the two cases are 31 and 41 runs, respectively?
L448: In an AMIP simulation, sea surface temperatures are specified, so ENSO (El Niño-Southern Oscillation) is not a suitable example in this context.
L456-461: Does this indicate that the tuned results are tied to a specific climate background?
L466-467: replace “equilibrium” with “energy balance”.
L471: Why are MSL, RSRC, and LRC difficult to tune?
L474: OSRC is not defined.
L476: TEMP@500 has been profoundly affected by tuning. Please explain the physical causes.
L479-480: Please add some discussion on how to tune the model performance for OLR and PRECIP.
L534-542: The 10-parameter case shows a larger difference in TOA outgoing shortwave flux (RSR) compared to the 20-parameter case relative to the default case (Fig. 4e and 6e). However, the 20-parameter case exhibits a larger difference in cloud compared to the 10-parameter case relative to the default case (Fig. 8d-e). Please explain this discrepancy.
L594-613: anomalies => biases.
L565-619: Does the coupled model directly utilize the optimized parameters from the AMIP simulations? If so, the TOA energy imbalance caused by the optimized parameters would eventually lead to climate drift in the long-term integration of the coupled model. This undermines the rationale and effectiveness of applying parameters tuned for an atmospheric model to an atmosphere-ocean coupled model. Meanwhile, a 2 W/m² energy imbalance at TOA is not a "slight energy imbalance" as stated in the abstract.
L767: forecasts -> prediction.
Citation: https://doi.org/10.5194/egusphere-2024-3770-RC2
Data sets
Model Optimization Simon Tett and Wenjun Liang https://doi.org/10.5281/zenodo.14772250
Viewed
HTML | XML | Total | Supplement | BibTeX | EndNote | |
---|---|---|---|---|---|---|
239 | 61 | 14 | 314 | 24 | 15 | 16 |
- HTML: 239
- PDF: 61
- XML: 14
- Total: 314
- Supplement: 24
- BibTeX: 15
- EndNote: 16
Viewed (geographical distribution)
Country | # | Views | % |
---|
Total: | 0 |
HTML: | 0 |
PDF: | 0 |
XML: | 0 |
- 1