the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 11 Apr 2024
fair-calibrate v1.4.1: calibration, constraining and validation of the FaIR simple climate model for reliable future climate projections
Abstract. Simple climate models (also known as emulators) have re-emerged as critical tools for analysis of climate policy. Emulators are efficient and highly parameterised, where the parameters are tunable to produce a diversity of global mean surface temperature (GMST) response pathways to a given emissions scenario. Only a small fraction of possible parameter combinations will produce historically consistent climate hindcasts, a necessary condition for trust in future projections. Alongside historical GMST, additional observed (e.g. ocean heat content) and emergent climate metrics (such as the equilibrium climate sensitivity) can be used as constraints upon the parameter sets used for climate projections. This paper describes a multi-variable constraining package for the FaIR simple climate model (FaIR versions 2.1.0 onwards) using a Bayesian framework. The steps are firstly to generate prior distributions of parameters for FaIR based on Coupled Model Intercomparison Project (CMIP6) Earth System models or Intergovernmental Panel on Climate Change (IPCC) assessed ranges, secondly to generate a large Monte Carlo prior ensemble of parameters to run FaIR with, and thirdly to produce a posterior set of parameters constrained on several observable and assessed climate metrics. Different calibrations can be produced for different emissions datasets or observed climate constraints, allowing version-controlled and continually updated calibrations to be produced. We show that two very different future projections to a given emission scenario can be obtained using emissions from the IPCC Sixth Assessment Report (AR6) (fair-calibrate v1.4.0) and from updated emissions datasets through 2022 (fair-calibrate v1.4.1) for similar climate constraints in both cases. fair-calibrate can be reconfigured for different source emissions datasets or target climate distributions, and new versions will be produced upon availability of new climate system data.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(1449 KB)
-
Supplement
(368 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(1449 KB) - Metadata XML
-
Supplement
(368 KB) - BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-708', Anonymous Referee #1, 20 May 2024
Paper provides rather detail descriptions of used climate model and parameter calibration procedure.
However, in my view it requires some clarifications and revision before publication.
Authors say in Abstract:
“We show that two very different future projections to a given emission scenario can be obtained using emissions from the IPCC Sixth Assessment Report (AR6) (fair-calibrate v1.4.0) and from updated emissions datasets through 2022 (fair-calibrate v1.4.1) for similar climate constraints in both cases.”
However, simulations presented in the paper were done not only using different parameters distributions but also very different emission scenarios (Fig 11). Moreover, in section 4 authors mentioned difference in future emission while explaining difference between results of v1.4.1 and v1.4.0 ensembles.
I think simulations with v1.4.0 set of model parameters need to be redone with different emissions scenarios. As I understand, historical simulations used to derive final parameters distributions were run until 2022 using historical plus SSP2-4.5 from AR6. It looks logical to produce another set of harmonized SSP emissions using 2022 SSP2-4.5 emissions instead of historical and repeat v1.4.0 ensemble with these emissions. It should make comparison cleaner. It also would be interesting to mention distribution of which model parameters are most sensitive to changing historical emissions.
Specific comments.
Page 7. Line 185. Reference to fig S1 is confusing.
Page 10 line 254.
concentrations of methane are in approximate equilibrium with pre-industrial concentrations.
It should be:
concentrations of methane are in approximate equilibrium with pre-industrial emissions.
Page 11
line 267 should be fig. S3,
line 269 should be fig S4b
Lines 271-273
“As 1750 emissions are subtracted from the total to report changes away from a pre-industrial equilibrium, the change in emissions (1750–2022) in v1.4.1 from PRIMAP-Hist is smaller than in v1.4.0, leading to longer atmospheric lifetimes necessary to reproduce concentrations.”
According to Fig. S4b, 1750 CH4 emissions are larger in v1.4.1 than in v1.4.0. If 1750 CH4 concentration are identical in both cases, it will require large CH4 lifetime is for v1.4.1 for concentration to be in equilibrium with emissions. Please comment.
Page 20.
“841 is one more than a highly composite number and allows many quantiles of the full distribution to correspond to a single ensemble member at each point in time.”
Needs explanation.
Page 21. Line 464
“The disagreement in the upper bound of ERFaci is large in absolute terms but small in relative terms.”
It seems to be other way around.
Page 21. Line 477
“Firstly, concentration (not emissions) driven runs were used to derive the IPCC warming ranges, which excludes the impact of carbon cycle sensitivity uncertainty on a future spread in CO2 concentrations and thus over-constraining the uncertainty range.”
IPCC ranges for SSP scenarios, shown in tables 5 and S4, are not from AR6 Table 4.2, which shows ranges based on CMIP6 model simulations, but from Table 4.5, which shows Assessment results for 20-year averaged GSAT change, based on multiple lines of evidence.
Citation: https://doi.org/10.5194/egusphere-2024-708-RC1 -
RC2: 'Comment on egusphere-2024-708', Anonymous Referee #2, 02 Jun 2024
The study proposes a new version of the FaIR simple climate model. It uses a Bayesian framework to estimate the parameters of the FaIR model and clearly explains the differences between fair-calibrate v1.4.0 and fair-calibrate v1.4.1. I enjoyed studying the entire manuscript. I have a few comments listed below.
- Can you please provide a flowchart of the working principle of fair-calibrate v1.4.1?
- If I understood correctly, the authors have used kernel density estimation as a prior distribution in the Bayesian framework. KDE can be very sensitive to outliers and data sparsity. Could you please discuss this in the manuscript?
- Can you please explain how you preserve the correlation structure between parameters while sampling parameters (section 3.2)?
- Can you please provide a table indicating the prior distribution of different parameters and the parameters of prior distributions?
- Can you please comment on the sensitivity of the parameters sampled using the MCMC approach?
- Is there any reason why you selected gaussian distribution (section 3.3.2)? Have you performed any goodness of fit tests?
- The authors have used RMSE as a constraint to reject ensemble members. Is it possible to use correlation as a metric to select ensemble members? Different metrics target different errors, and restricting it to a single metric might lose some information.
Citation: https://doi.org/10.5194/egusphere-2024-708-RC2 - AC1: 'Response to reviewers', Christopher Smith, 05 Jul 2024
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-708', Anonymous Referee #1, 20 May 2024
Paper provides rather detail descriptions of used climate model and parameter calibration procedure.
However, in my view it requires some clarifications and revision before publication.
Authors say in Abstract:
“We show that two very different future projections to a given emission scenario can be obtained using emissions from the IPCC Sixth Assessment Report (AR6) (fair-calibrate v1.4.0) and from updated emissions datasets through 2022 (fair-calibrate v1.4.1) for similar climate constraints in both cases.”
However, simulations presented in the paper were done not only using different parameters distributions but also very different emission scenarios (Fig 11). Moreover, in section 4 authors mentioned difference in future emission while explaining difference between results of v1.4.1 and v1.4.0 ensembles.
I think simulations with v1.4.0 set of model parameters need to be redone with different emissions scenarios. As I understand, historical simulations used to derive final parameters distributions were run until 2022 using historical plus SSP2-4.5 from AR6. It looks logical to produce another set of harmonized SSP emissions using 2022 SSP2-4.5 emissions instead of historical and repeat v1.4.0 ensemble with these emissions. It should make comparison cleaner. It also would be interesting to mention distribution of which model parameters are most sensitive to changing historical emissions.
Specific comments.
Page 7. Line 185. Reference to fig S1 is confusing.
Page 10 line 254.
concentrations of methane are in approximate equilibrium with pre-industrial concentrations.
It should be:
concentrations of methane are in approximate equilibrium with pre-industrial emissions.
Page 11
line 267 should be fig. S3,
line 269 should be fig S4b
Lines 271-273
“As 1750 emissions are subtracted from the total to report changes away from a pre-industrial equilibrium, the change in emissions (1750–2022) in v1.4.1 from PRIMAP-Hist is smaller than in v1.4.0, leading to longer atmospheric lifetimes necessary to reproduce concentrations.”
According to Fig. S4b, 1750 CH4 emissions are larger in v1.4.1 than in v1.4.0. If 1750 CH4 concentration are identical in both cases, it will require large CH4 lifetime is for v1.4.1 for concentration to be in equilibrium with emissions. Please comment.
Page 20.
“841 is one more than a highly composite number and allows many quantiles of the full distribution to correspond to a single ensemble member at each point in time.”
Needs explanation.
Page 21. Line 464
“The disagreement in the upper bound of ERFaci is large in absolute terms but small in relative terms.”
It seems to be other way around.
Page 21. Line 477
“Firstly, concentration (not emissions) driven runs were used to derive the IPCC warming ranges, which excludes the impact of carbon cycle sensitivity uncertainty on a future spread in CO2 concentrations and thus over-constraining the uncertainty range.”
IPCC ranges for SSP scenarios, shown in tables 5 and S4, are not from AR6 Table 4.2, which shows ranges based on CMIP6 model simulations, but from Table 4.5, which shows Assessment results for 20-year averaged GSAT change, based on multiple lines of evidence.
Citation: https://doi.org/10.5194/egusphere-2024-708-RC1 -
RC2: 'Comment on egusphere-2024-708', Anonymous Referee #2, 02 Jun 2024
The study proposes a new version of the FaIR simple climate model. It uses a Bayesian framework to estimate the parameters of the FaIR model and clearly explains the differences between fair-calibrate v1.4.0 and fair-calibrate v1.4.1. I enjoyed studying the entire manuscript. I have a few comments listed below.
- Can you please provide a flowchart of the working principle of fair-calibrate v1.4.1?
- If I understood correctly, the authors have used kernel density estimation as a prior distribution in the Bayesian framework. KDE can be very sensitive to outliers and data sparsity. Could you please discuss this in the manuscript?
- Can you please explain how you preserve the correlation structure between parameters while sampling parameters (section 3.2)?
- Can you please provide a table indicating the prior distribution of different parameters and the parameters of prior distributions?
- Can you please comment on the sensitivity of the parameters sampled using the MCMC approach?
- Is there any reason why you selected gaussian distribution (section 3.3.2)? Have you performed any goodness of fit tests?
- The authors have used RMSE as a constraint to reject ensemble members. Is it possible to use correlation as a metric to select ensemble members? Different metrics target different errors, and restricting it to a single metric might lose some information.
Citation: https://doi.org/10.5194/egusphere-2024-708-RC2 - AC1: 'Response to reviewers', Christopher Smith, 05 Jul 2024
Peer review completion
Journal article(s) based on this preprint
Data sets
FaIR calibration data Chris Smith https://zenodo.org/records/10566813
Model code and software
fair-calibrate Chris Smith https://github.com/chrisroadmap/fair-calibrate/tree/v1.4.1
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 763 | 483 | 42 | 1,288 | 85 | 35 | 30 |
- HTML: 763
- PDF: 483
- XML: 42
- Total: 1,288
- Supplement: 85
- BibTeX: 35
- EndNote: 30
Viewed (geographical distribution)
| Country | # | Views | % |
|---|---|---|---|
| United States of America | 1 | 517 | 39 |
| United Kingdom | 2 | 177 | 13 |
| Germany | 3 | 102 | 7 |
| Netherlands | 4 | 64 | 4 |
| France | 5 | 54 | 4 |
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
- 517
Cited
1 citations as recorded by crossref.
Donald P. Cummins
Hege-Beate Fredriksen
Zebedee Nicholls
Malte Meinshausen
Myles Allen
Stuart Jenkins
Nicholas Leach
Camilla Mathison
Antti-Ilari Partanen
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(1449 KB) - Metadata XML
-
Supplement
(368 KB) - BibTeX
- EndNote
- Final revised paper