Testing the assumptions in emergent constraints: Why does the 'Emergent constraint on equilibrium climate sensitivity from global temperature variability' work for CMIP5 and not CMIP6?

Williamson, Mark S.; Cox, Peter M.; Huntingford, Chris; Nijsse, Femke J. M. M.

doi:https://doi.org/10.5194/egusphere-2023-1093

Preprints

https://doi.org/10.5194/egusphere-2023-1093

Preprints

30 May 2023

| 30 May 2023

Testing the assumptions in emergent constraints: Why does the 'Emergent constraint on equilibrium climate sensitivity from global temperature variability' work for CMIP5 and not CMIP6?

Mark S. Williamson, Peter M. Cox, Chris Huntingford, and Femke J. M. M. Nijsse

Abstract. It was shown that a theoretically derived relation between annual global mean temperature variability and climate sensitivity held in the (then latest) state-of-the-art CMIP5 climate model ensemble (Cox et al (2018), hereafter CHW18). This so called emergent relationship was then used with observations to constrain the value of equilibrium climate sensitivity (ECS) to about 3 °C. Since this study was published, CMIP6, a newer ensemble of climate models has become available. Schlund et al (2020) showed that many of the emergent constraints found in CMIP5 were much weaker in the newer ensemble including that of CHW18. As the constraint in CHW18 was based on a relationship derived from reasonable physical principles it is of interest to find out why it got weaker in CMIP6. Here, we look in detail at the assumptions made in deriving the emergent relationship in CHW18 and test them for CMIP5 and CMIP6 models. We show one assumption, that of low correlation and variation between the internal variability parameter and ECS, while true for CMIP5 is not true for CMIP6. When accounted for, an emergent relationship appears once again in both CMIP ensembles implying the theoretical basis is still applicable although the original assumption in CHW18 does not. Unfortunately however, we are unable to provide an emergent constraint in CMIP6 as observational estimates of the internal variability parameter are too uncertain.

Received: 23 May 2023 – Discussion started: 30 May 2023

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.

Download & links

Preprint (PDF, 877 KB)

Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
Preprint (877 KB)

Download & links

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Journal article(s) based on this preprint

11 Jul 2024

Testing the assumptions in emergent constraints: why does the “emergent constraint on equilibrium climate sensitivity from global temperature variability” work for CMIP5 and not CMIP6?

Mark S. Williamson, Peter M. Cox, Chris Huntingford, and Femke J. M. M. Nijsse

Earth Syst. Dynam., 15, 829–852, https://doi.org/10.5194/esd-15-829-2024,https://doi.org/10.5194/esd-15-829-2024, 2024

Short summary

Mark S. Williamson, Peter M. Cox, Chris Huntingford, and Femke J. M. M. Nijsse

Interactive discussion

Status: closed

RC1:
'Comment on egusphere-2023-1093', BB Cael, 04 Jul 2023

This manuscript addresses an important issue with a discrepancy between an emergent constraint that held well for CMIP5 and less so for CMIP6. The authors identify the reason, relating to an assumption in the emergent constraint's derivation.
I found this to be a very useful, interesting, and convincing paper; I cannot remember the last time, if ever, that I had so few comments on a manuscript. Nice work.
Minor Comments:
It would have been nice to explain a bit more in the abstract what 'the internal variability parameter' is, if there is space.
It would have been nice to see Spearman or Kendall correlations & also p-values on these correlations, though I understand why the authors chose to stick with the Pearson correlation that's in the literature being compared to.
Even More Minor Comments:
Line 2 – suggest deleting ‘(then latest) state-of-the-art’ – sounds impartial

Line 7 – would say ‘is weaker’ not ‘got weaker’

Line 64 – incomplete sentence. In general lines 64-69 could use some rephrasing.

Line 161 – uncorrelated with what?

Line 268 – would replace ‘good’ with ‘sound’

Line 272 – would replace ‘good’ with ‘strong’
Thanks for the enjoyable read.
B. B. Cael

Citation: https://doi.org/10.5194/egusphere-2023-1093-RC1
- AC1: 'Reply on RC1', mark williamson, 15 Dec 2023
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-1093/egusphere-2023-1093-AC1-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2023-1093-AC1
RC2:
'Comment on egusphere-2023-1093', Anonymous Referee #2, 20 Sep 2023
The manuscript
"Testing the assumptions in emergent constraints; Why does the 'Emergent constraint on equilibrium climate sensitivity from global temperature variability' work for CMIP5 and not CMIP6? by Mark S. Williamson, Peter M. Cox, Chris Huntingford and Femke J. M. M. Nijsse
reviews a standard protocol for the derivation of the equilibriium climate sensitivity (ECS) from annual global mean temperature variability, that was introduced successfully on CMIP5 multimodel ensemble in 2018 (Cox et al. 2018). This emergent constraint proved to be much less robust in CMIP6 multimodel ensemble wrt. CMIP5, and this work addresses specifically the main assumptions underlying it, in order to identify what is not holding anymore in CMIP6 models.
Overall, I acknowledge that the work has a robust, logical structure: the methodology is well established, some important additions, such as testing the 2-boxes conceptual model provide better context for the results obtained with the simpler Hasselmann's model. I do think, as well, that the manuscript falls a bit short in providing an explanation for the reasons why CMIP6 models do not exhibit such a strong emergent constraint as the one found in CMIP5 ensembles. In doing so, I struggle to see how the work could contribute to broader discussions on what use one can make of emergent constraints in the context of detection of the forced signal and attribution. Therefore, I think that the manuscript could substantially improve if at least the following two aspects would be taken into account.
Why is the internal variability parameter so crucial in defining the ECS emergent constraint in CMIP6. Why is it not the case for CMIP5?

Is the ECS-Ψ constraint peculiar? What happens with other notable emergent constraints? Do they also differ from CMIP5? If so, do they differ because of similar reasons?

I believe that the first point is of particular relevance, given its implications for the development of synthetic model diagnostics and the usage of historical/paleoclimatic evidences to better understand and predict future climate scenarios.

SPECIFIC COMMENTS
ll. 56-57: some reference on the usage of simple models for the reproduction of forced global mean temperature response could be useful here and elsewhere in the manuscript;

ll. 101-104: I do not have clear if the choice of 15 CMIP5 models is just guided by the need for consistency with Geoffrey et al. 2013 work or if there are other practical/theoretical reasons for that. Given that to the best of my knowledge more CMIP5 models should be available, I wonder if it would be possible to have a similar amount of models in both CMIP5 and CMIP6 ensembles. If that is not possible, I wonder if some arguments could be provided on the implications of the size of the ensemble for the robustness of the discussed relation.

ll. 105-106: a bit in line with my previous comment, I see that it is a common procedure limiting to r1i1p1 or r1i1p1f1 runs, but it might be worthwhile, given that the interpretation strongly relies on the retrieval of the internal variability parameter, to discuss a bit if such parameter holds across different ensemble runs. It is not entirely straightforward to me, whether the choice of the parameter in the chosen runs would be representative of the other runs as well;

ll. 155-156: provided the discussion above, I do not have clear why the authors opt for linearly detrending the temperature with the 55 years moving window, especially given that in a previous paper (Cox et al. 2018b) they noticed that retaining the external forcing would possibly improve the emergent relationship;

l. 160: if the correlation value is meant to be the one in the title of the panels of in Figure 2, it would be useful to explicitly mention it here;

Figures 4 and 6: these figures, showing the relations between the two assumed parameters in CMIP5 and CMIP6, do not seem to add more arguments to the discussion than what already mentioned in the text. Consider whether is possible to remove them;

Figures 9 and 10: when comparing the PDFs for piControl and historical, the authors evidence their similarities. It is a bit overlooked, though, that at first glance CMIP5 and CMIP6 in the historical runs exhibit substantially different medians and variance. Furthermore, the median for piControl in CMIP6 is possibly negative, whereas it is positive in CMIP5. Can the authors provide an explanation for that?

l. 288: the authors acknowledge that this remains an unanswered question, but I do think it is crucial to try to provide even a speculative explanation for that, in order to improve the usability of the main result described in the manuscript, at least something that could serve as triggering hypothesis for future work;

TECHNICAL CORRECTIONS
l. 67: for ‘all practical purposes’ -> ‘for all practical purposes’;

l. 175: is it actually -0.06, rather than -0.60;

l. 226: “results” -> “result”;

l. 238: “RH” -> “rhs”;

Figures 9 and 10: the legends do not seem to agree with the caption and the text;
Citation: https://doi.org/10.5194/egusphere-2023-1093-RC2
- AC1: 'Reply on RC1', mark williamson, 15 Dec 2023
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-1093/egusphere-2023-1093-AC1-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2023-1093-AC1

Interactive discussion

Status: closed

RC1:
'Comment on egusphere-2023-1093', BB Cael, 04 Jul 2023

This manuscript addresses an important issue with a discrepancy between an emergent constraint that held well for CMIP5 and less so for CMIP6. The authors identify the reason, relating to an assumption in the emergent constraint's derivation.
I found this to be a very useful, interesting, and convincing paper; I cannot remember the last time, if ever, that I had so few comments on a manuscript. Nice work.
Minor Comments:
It would have been nice to explain a bit more in the abstract what 'the internal variability parameter' is, if there is space.
It would have been nice to see Spearman or Kendall correlations & also p-values on these correlations, though I understand why the authors chose to stick with the Pearson correlation that's in the literature being compared to.
Even More Minor Comments:
Line 2 – suggest deleting ‘(then latest) state-of-the-art’ – sounds impartial

Line 7 – would say ‘is weaker’ not ‘got weaker’

Line 64 – incomplete sentence. In general lines 64-69 could use some rephrasing.

Line 161 – uncorrelated with what?

Line 268 – would replace ‘good’ with ‘sound’

Line 272 – would replace ‘good’ with ‘strong’
Thanks for the enjoyable read.
B. B. Cael

Citation: https://doi.org/10.5194/egusphere-2023-1093-RC1
- AC1: 'Reply on RC1', mark williamson, 15 Dec 2023
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-1093/egusphere-2023-1093-AC1-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2023-1093-AC1
RC2:
'Comment on egusphere-2023-1093', Anonymous Referee #2, 20 Sep 2023
The manuscript
"Testing the assumptions in emergent constraints; Why does the 'Emergent constraint on equilibrium climate sensitivity from global temperature variability' work for CMIP5 and not CMIP6? by Mark S. Williamson, Peter M. Cox, Chris Huntingford and Femke J. M. M. Nijsse
reviews a standard protocol for the derivation of the equilibriium climate sensitivity (ECS) from annual global mean temperature variability, that was introduced successfully on CMIP5 multimodel ensemble in 2018 (Cox et al. 2018). This emergent constraint proved to be much less robust in CMIP6 multimodel ensemble wrt. CMIP5, and this work addresses specifically the main assumptions underlying it, in order to identify what is not holding anymore in CMIP6 models.
Overall, I acknowledge that the work has a robust, logical structure: the methodology is well established, some important additions, such as testing the 2-boxes conceptual model provide better context for the results obtained with the simpler Hasselmann's model. I do think, as well, that the manuscript falls a bit short in providing an explanation for the reasons why CMIP6 models do not exhibit such a strong emergent constraint as the one found in CMIP5 ensembles. In doing so, I struggle to see how the work could contribute to broader discussions on what use one can make of emergent constraints in the context of detection of the forced signal and attribution. Therefore, I think that the manuscript could substantially improve if at least the following two aspects would be taken into account.
Why is the internal variability parameter so crucial in defining the ECS emergent constraint in CMIP6. Why is it not the case for CMIP5?

Is the ECS-Ψ constraint peculiar? What happens with other notable emergent constraints? Do they also differ from CMIP5? If so, do they differ because of similar reasons?

I believe that the first point is of particular relevance, given its implications for the development of synthetic model diagnostics and the usage of historical/paleoclimatic evidences to better understand and predict future climate scenarios.

SPECIFIC COMMENTS
ll. 56-57: some reference on the usage of simple models for the reproduction of forced global mean temperature response could be useful here and elsewhere in the manuscript;

ll. 101-104: I do not have clear if the choice of 15 CMIP5 models is just guided by the need for consistency with Geoffrey et al. 2013 work or if there are other practical/theoretical reasons for that. Given that to the best of my knowledge more CMIP5 models should be available, I wonder if it would be possible to have a similar amount of models in both CMIP5 and CMIP6 ensembles. If that is not possible, I wonder if some arguments could be provided on the implications of the size of the ensemble for the robustness of the discussed relation.

ll. 105-106: a bit in line with my previous comment, I see that it is a common procedure limiting to r1i1p1 or r1i1p1f1 runs, but it might be worthwhile, given that the interpretation strongly relies on the retrieval of the internal variability parameter, to discuss a bit if such parameter holds across different ensemble runs. It is not entirely straightforward to me, whether the choice of the parameter in the chosen runs would be representative of the other runs as well;

ll. 155-156: provided the discussion above, I do not have clear why the authors opt for linearly detrending the temperature with the 55 years moving window, especially given that in a previous paper (Cox et al. 2018b) they noticed that retaining the external forcing would possibly improve the emergent relationship;

l. 160: if the correlation value is meant to be the one in the title of the panels of in Figure 2, it would be useful to explicitly mention it here;

Figures 4 and 6: these figures, showing the relations between the two assumed parameters in CMIP5 and CMIP6, do not seem to add more arguments to the discussion than what already mentioned in the text. Consider whether is possible to remove them;

Figures 9 and 10: when comparing the PDFs for piControl and historical, the authors evidence their similarities. It is a bit overlooked, though, that at first glance CMIP5 and CMIP6 in the historical runs exhibit substantially different medians and variance. Furthermore, the median for piControl in CMIP6 is possibly negative, whereas it is positive in CMIP5. Can the authors provide an explanation for that?

l. 288: the authors acknowledge that this remains an unanswered question, but I do think it is crucial to try to provide even a speculative explanation for that, in order to improve the usability of the main result described in the manuscript, at least something that could serve as triggering hypothesis for future work;

TECHNICAL CORRECTIONS
l. 67: for ‘all practical purposes’ -> ‘for all practical purposes’;

l. 175: is it actually -0.06, rather than -0.60;

l. 226: “results” -> “result”;

l. 238: “RH” -> “rhs”;

Figures 9 and 10: the legends do not seem to agree with the caption and the text;
Citation: https://doi.org/10.5194/egusphere-2023-1093-RC2
- AC1: 'Reply on RC1', mark williamson, 15 Dec 2023
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-1093/egusphere-2023-1093-AC1-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2023-1093-AC1

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision | EF: Editorial file upload

ED: Reconsider after major revisions (05 Jan 2024) by Roberta D'Agostino

AR by mark williamson on behalf of the Authors (20 Feb 2024) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (29 Feb 2024) by Roberta D'Agostino

RR by Anonymous Referee #3 (04 Apr 2024)

Suggestions for revision or reasons for rejection

Testing the assumptions in emergent constraints: Why does the ‘Emergent constraint on equilibrium climate sensitivity from global temperature variability’ work for CMIP5 and not CMIP6?

M. S. Williamson, P. M. Cox, C. Huntingsford, F. J. M. M. Nijsse

Cox et al. (2018) developed a physically-motivated scaling between a metric of global temperature variability and climate sensitivity. When applied to CMIP5 climate models, the researchers verified that the emergent relationship was strong and suggested an ECS value of ~3K. In subsequent work, other researchers used data from CMIP6 models and found that the emergent relationship essentially vanished. It has been unclear why. This manuscript investigates this open question by scrutinizing key assumptions in the original Cox et al. (2018) paper. The authors determine that the change in the strength of the emergent relationship between CMIP5 and CMIP6 is related to an assumption that ignores a term (related to “weather forcing” variability) in their underlying physical model. When this term is re-incorporated into their emergent relationship, the authors find that the emergent relationship is quite strong for both CMIP5 and CMIP5 climate models. The authors also include analysis to test the robustness of their results to sampling (within CMIP model ensembles) and whether the results hold in simpler energy balance models (the results hold reasonably well).

This manuscript is well-motivated, well-written, and interesting. I appreciate the author’s effort in investigating this curious issue and the thoroughness of the analysis. My comments are generally minor suggestions that are meant to clarify or improve the presentation to the reader. I recommend the manuscript for publication.

Lines 7 / 52: Schlund et al find an r^2 value of 0.01 in CMIP6. The language here suggests that there was still a (weaker) constraint, but it seems like the constraint (as used in Cox et al) had disappeared / was non-existent in CMIP6.

Line 17: I think the goal is not just to make climate models “look increasingly realistic” but to make climate models that are actually more realistic representations of Earth’s climate

Lines 24 - 25: Isn’t this the “likely range”?

Line 54: Note that you haven’t actually explained the physical underpinnings of the emergent constraint yet, so you are asserting that it has reasonable physical principles here (or relying on the reader to be familiar with Cox et al 2018).

Line 61: Is it true that temperature observations are “relatively un-autocorrelated”? I assume you mean in time? This strikes me as untrue. Could this be qualified to make it true? Or, alternatively, you could point to evidence that the observations are sufficiently reliable.

Line 89: Should this be “…designed to model the *impact of* random, fast internally…”

Line 95: The CMIP5 historical experiment ended in 2005. Were these extended with another experiment?

Lines 96 - 97: I’m not sure where it would make sense to add this, but some studies indicated that the correlation coefficient was lower when more CMIP5 models were added to the original Cox et al. (2018) analysis (Po-Chedley et al. comment showed a correlation of r = 0.54 and Schlund et al. showed a correlation of r = 0.56). This slightly complicates the story a little bit (it isn’t solely the CMIP6 ensemble that showed a weaker constraint). You might be able to allude to this in the paragraph at lines 113 to 116 (and you could note that sigma_Q improves the emergent relationship in both CMIP5 and CMIP6 later on in the manuscript).

Eq. 4: I had thought that you might get sigma_Q by analyzing variations in the TOA flux. I now realize that would be problematic because TOA balance includes the climate response (lambda * T) and isn’t a pure estimate of “weather forcing”? Regardless, was this formula in Cox et al (2018)? Or a follow-up Williamson et al. paper? If so, could you cite them? I don’t have great physical intuition for this equation, so providing more context (in text or via citations) would be helpful.

Line 191: Is this expected because this was the case in Cox et al? If so, you could signal this by noting: “Consistent with Cox et al. (2018), …”

Lines 195 - 196: I’m a little unclear about what exactly this sentence is saying. Could you just explicitly write out that Q_(2xCO2)/sigma_Q is correlated with ECS in CMIP6 (I assume this is what you mean)?

Figures 2 - 4: Would it make sense to combine these figures into a single figure (with 6 subplots)? They seem sufficiently related that it would be helpful to put them together. This comment could also apply separately to Figures 5 and 6.

Lines 201 - 203: It would be useful to plot the right hand side of Eq. 2 versus ECS and the 1-to-1 line so that the reader can see how accurately this equation predicts ECS in CMIP models (to be clear, I don’t expect it to be perfect, but this would be a natural plot to include somewhere).

Line 353: I suggest being more explicit: “namely simga_Q is correlated with ECS.”

Hide

RR by Anonymous Referee #2 (05 Apr 2024)

ED: Publish subject to minor revisions (review by editor) (05 Apr 2024) by Roberta D'Agostino

AR by mark williamson on behalf of the Authors (19 Apr 2024) Author's response Author's tracked changes Manuscript

ED: Publish as is (17 May 2024) by Roberta D'Agostino

AR by mark williamson on behalf of the Authors (21 May 2024) Manuscript

Journal article(s) based on this preprint

11 Jul 2024

Testing the assumptions in emergent constraints: why does the “emergent constraint on equilibrium climate sensitivity from global temperature variability” work for CMIP5 and not CMIP6?

Mark S. Williamson, Peter M. Cox, Chris Huntingford, and Femke J. M. M. Nijsse

Earth Syst. Dynam., 15, 829–852, https://doi.org/10.5194/esd-15-829-2024,https://doi.org/10.5194/esd-15-829-2024, 2024

Short summary

Mark S. Williamson, Peter M. Cox, Chris Huntingford, and Femke J. M. M. Nijsse

Viewed

Total article views: 725 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
474	213	38	725	30	27

HTML: 474
PDF: 213
XML: 38
Total: 725
BibTeX: 30
EndNote: 27

Views and downloads (calculated since 30 May 2023)

Month	HTML	PDF	XML	Total
May 2023	52	12	1	65
Jun 2023	71	66	5	142
Jul 2023	60	22	5	87
Aug 2023	21	8	0	29
Sep 2023	42	16	2	60
Oct 2023	22	16	2	40
Nov 2023	13	12	2	27
Dec 2023	29	15	6	50
Jan 2024	10	4	2	16
Feb 2024	17	11	0	28
Mar 2024	25	5	3	33
Apr 2024	24	4	3	31
May 2024	28	17	3	48
Jun 2024	53	5	4	62
Jul 2024	7	0	7
Aug 2024	0

Cumulative views and downloads (calculated since 30 May 2023)

Month	HTML	PDF	XML	Total
May 2023	52	12	1	65
Jun 2023	71	66	5	142
Jul 2023	60	22	5	87
Aug 2023	21	8	0	29
Sep 2023	42	16	2	60
Oct 2023	22	16	2	40
Nov 2023	13	12	2	27
Dec 2023	29	15	6	50
Jan 2024	10	4	2	16
Feb 2024	17	11	0	28
Mar 2024	25	5	3	33
Apr 2024	24	4	3	31
May 2024	28	17	3	48
Jun 2024	53	5	4	62
Jul 2024	7	0	7
Aug 2024	0

Viewed (geographical distribution)

Total article views: 696 (including HTML, PDF, and XML) Thereof 696 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 01 Sep 2024

Download

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Preprint (877 KB)
Metadata XML

Short summary

Emergent constraints on equilibrium climate sensitivity (ECS) have generally got statistically weaker in the latest set of state-of-the-art climate models (CMIP6) compared to past sets (CMIP5). We look at why this weakening happened for one particular study (Cox et al, 2018) and attribute it to an assumption made in the theory that when corrected for restores a stronger relationship between predictor and ECS.


Total:	0
HTML:	0
PDF:	0
XML:	0