the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
An Extensible Perturbed Parameter Ensemble (PPE) for the Community Atmosphere Model Version 6
Abstract. This paper documents the methodology and preliminary results from a Perturbed Parameter Ensemble (PPE) technique, where multiple parameters are varied simultaneously and the parameter values are determined with Latin hypercube sampling. This is done with the Community Atmosphere Model version 6 (CAM6), the atmospheric component of the Community Earth System Model version 2 (CESM2). We apply the PPE method to CESM2-CAM6 to understand climate sensitivity to atmospheric physics parameters. The initial simulations vary 45 parameters in the microphysics, convection, turbulence and aerosol schemes with 263 ensemble members. These atmospheric parameters are typically the most uncertain in many climate models. Control simulations are analyzed and targeted simulations to understand climate forcing due to aerosols and fast climate feedbacks. The use of various emulators is explored in the multi- dimensional space mapping input parameters to output metrics. Parameter impacts on various model outputs, such as radiation, cloud and aerosol properties are evaluated. Machine learning is also used to probe optimal parameter values against observations. Our findings show that using PPE is a valuable tool for climate uncertainty analysis. Furthermore, by varying many parameters simultaneously, we find that many different combinations of parameter values can produce results consistent with observations, and thus careful analysis of tuning is important. The CESM2-CAM6 PPE is publicly available, and extensible to other configurations to address questions of other model processes in the atmosphere and other model components (e.g. coupling to the land surface).
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(15141 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(15141 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2023-2165', Anonymous Referee #1, 12 Feb 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2023-2165/egusphere-2023-2165-RC1-supplement.pdf
- AC3: 'Reply on RC1', Trude Eidhammer, 05 Jul 2024
-
CC1: 'Comment on egusphere-2023-2165', Pappu Paul, 30 Apr 2024
In this article there is nothing mentioned about which data sets was used for Table 2 & 3 i.e., is it for present day simulations or 4k SST or the preindustrial ? For a class project I tried to reproduce the tables and also found you did not mention about exactly which kernel (constant or RBF or anything else) was used in this study for GP emulation? What was the length-scale and variance ?
Citation: https://doi.org/10.5194/egusphere-2023-2165-CC1 -
AC1: 'Reply on CC1', Trude Eidhammer, 06 May 2024
Thank you for the comments.
The Emulator results are all conducted with the present-day PPE. We will clarify this in the revised manuscript.
For the GP emulation we used the RBF kernel, and initialized with noise_variance = 1 and lengthscales=1, which are the default in the ESMF. However, these values are learnt when we fit the GP
Citation: https://doi.org/10.5194/egusphere-2023-2165-AC1 -
CC2: 'Reply on AC1', Pappu Paul, 06 May 2024
Thank you for the reply.
I tried use RBF kernel with lengthscales=1 and noise_variance = 1 in GP emulation. But all my predicted output values are zero. Note, when I use constant kernel I got non zero values.
Here are my X and Y data sets:
X_train_Values= (211, 46), X_test_Values= (52, 46), Y_train_Values= (211, 192, 288), Y_test_Values= (52, 192, 288)Is it the correct way to arrange data sets? or in this study you processed the data sets in different way to feed the emulator?
Citation: https://doi.org/10.5194/egusphere-2023-2165-CC2
-
CC2: 'Reply on AC1', Pappu Paul, 06 May 2024
-
AC1: 'Reply on CC1', Trude Eidhammer, 06 May 2024
-
RC2: 'Comment on egusphere-2023-2165', Anonymous Referee #2, 23 May 2024
This is an easy-to-read and useful introduction to NCAR’s perturbed parameter ensemble (PPE) methodology. The results are interesting and important. However, I think that some additional discussion by the authors could clarify the interpretation of the results.
Major comments:
Lines 77–78: “we initially created 250 different sets of parameter values in addition to the default CESM2-CAM6 setup (total of 251 sets).” How was this number chosen? What is the consequence if only half of the 250 parameters sets are emulated? The paper contains a nice comparison of emulator techniques, but I wonder if an excellent emulator could be thwarted by a sparse sample of parameter sets.
Line 241: “First, we will illustrate the basic spread across the PPE for several key features of the simulated climate system.” Is this the ensemble spread over all values based on a uniform sample of parameter values within the expert-chosen ranges? Could you provide some comments on how these spreads should be interpreted? The magnitude of the spread would be expected to depend sensitively on the chosen parameter range. If that range is chosen subjectively, then the spread will inherit that subjectivity. In principle, the range might be problematic because, e.g., we know that some ensemble members produce an unrealistic climate, because the authors state on lines 409–410 that “First, we will illustrate the basic spread across the PPE for several key features of the simulated climate system”. Given this, are the spreads realistic? E.g., the max ice fall speed factor is 25 times the min value, suggesting that we don’t know the ice fall speed within an order of magnitude. Is this true? To cite another example, the accretion enhancing factor varies by a factor of 100. Is this degree of uncertainty realistic?
Minor comments:
Lines 44-46: “in the current study we perturb 45 different parameters, which would require a minimum of 3.5·10^13 (2^45) simulations using OAT if each parameter was tested with only two values in all combinations.” How are you defining “OAT” here? 2^45 sample points would fill the entire 45-dimensional volume of parameter space, which would involve perturbing all parameters simultaneously, not one at a time. Perturbing each parameter individually would yield only 2*45 samples, no?
Table 1: The max value of DCS is listed as 1.0e-6. Should it be 1000e-6?
Line 247: “global, annual mean net top of atmosphere flux balance (TOA: Figure 3H).” The definition of TOA is unclear to me. Does a net downward flux have positive TOA? Or negative TOA? Is TOA different than RESTOM (line 297)?
Line 414: Replace “relative” with “relatively”.
Line 417: Replace “Colombia” with “Venezuela”.
Citation: https://doi.org/10.5194/egusphere-2023-2165-RC2 -
AC2: 'Reply on RC2', Trude Eidhammer, 04 Jul 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2023-2165/egusphere-2023-2165-AC2-supplement.pdf
-
AC2: 'Reply on RC2', Trude Eidhammer, 04 Jul 2024
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2023-2165', Anonymous Referee #1, 12 Feb 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2023-2165/egusphere-2023-2165-RC1-supplement.pdf
- AC3: 'Reply on RC1', Trude Eidhammer, 05 Jul 2024
-
CC1: 'Comment on egusphere-2023-2165', Pappu Paul, 30 Apr 2024
In this article there is nothing mentioned about which data sets was used for Table 2 & 3 i.e., is it for present day simulations or 4k SST or the preindustrial ? For a class project I tried to reproduce the tables and also found you did not mention about exactly which kernel (constant or RBF or anything else) was used in this study for GP emulation? What was the length-scale and variance ?
Citation: https://doi.org/10.5194/egusphere-2023-2165-CC1 -
AC1: 'Reply on CC1', Trude Eidhammer, 06 May 2024
Thank you for the comments.
The Emulator results are all conducted with the present-day PPE. We will clarify this in the revised manuscript.
For the GP emulation we used the RBF kernel, and initialized with noise_variance = 1 and lengthscales=1, which are the default in the ESMF. However, these values are learnt when we fit the GP
Citation: https://doi.org/10.5194/egusphere-2023-2165-AC1 -
CC2: 'Reply on AC1', Pappu Paul, 06 May 2024
Thank you for the reply.
I tried use RBF kernel with lengthscales=1 and noise_variance = 1 in GP emulation. But all my predicted output values are zero. Note, when I use constant kernel I got non zero values.
Here are my X and Y data sets:
X_train_Values= (211, 46), X_test_Values= (52, 46), Y_train_Values= (211, 192, 288), Y_test_Values= (52, 192, 288)Is it the correct way to arrange data sets? or in this study you processed the data sets in different way to feed the emulator?
Citation: https://doi.org/10.5194/egusphere-2023-2165-CC2
-
CC2: 'Reply on AC1', Pappu Paul, 06 May 2024
-
AC1: 'Reply on CC1', Trude Eidhammer, 06 May 2024
-
RC2: 'Comment on egusphere-2023-2165', Anonymous Referee #2, 23 May 2024
This is an easy-to-read and useful introduction to NCAR’s perturbed parameter ensemble (PPE) methodology. The results are interesting and important. However, I think that some additional discussion by the authors could clarify the interpretation of the results.
Major comments:
Lines 77–78: “we initially created 250 different sets of parameter values in addition to the default CESM2-CAM6 setup (total of 251 sets).” How was this number chosen? What is the consequence if only half of the 250 parameters sets are emulated? The paper contains a nice comparison of emulator techniques, but I wonder if an excellent emulator could be thwarted by a sparse sample of parameter sets.
Line 241: “First, we will illustrate the basic spread across the PPE for several key features of the simulated climate system.” Is this the ensemble spread over all values based on a uniform sample of parameter values within the expert-chosen ranges? Could you provide some comments on how these spreads should be interpreted? The magnitude of the spread would be expected to depend sensitively on the chosen parameter range. If that range is chosen subjectively, then the spread will inherit that subjectivity. In principle, the range might be problematic because, e.g., we know that some ensemble members produce an unrealistic climate, because the authors state on lines 409–410 that “First, we will illustrate the basic spread across the PPE for several key features of the simulated climate system”. Given this, are the spreads realistic? E.g., the max ice fall speed factor is 25 times the min value, suggesting that we don’t know the ice fall speed within an order of magnitude. Is this true? To cite another example, the accretion enhancing factor varies by a factor of 100. Is this degree of uncertainty realistic?
Minor comments:
Lines 44-46: “in the current study we perturb 45 different parameters, which would require a minimum of 3.5·10^13 (2^45) simulations using OAT if each parameter was tested with only two values in all combinations.” How are you defining “OAT” here? 2^45 sample points would fill the entire 45-dimensional volume of parameter space, which would involve perturbing all parameters simultaneously, not one at a time. Perturbing each parameter individually would yield only 2*45 samples, no?
Table 1: The max value of DCS is listed as 1.0e-6. Should it be 1000e-6?
Line 247: “global, annual mean net top of atmosphere flux balance (TOA: Figure 3H).” The definition of TOA is unclear to me. Does a net downward flux have positive TOA? Or negative TOA? Is TOA different than RESTOM (line 297)?
Line 414: Replace “relative” with “relatively”.
Line 417: Replace “Colombia” with “Venezuela”.
Citation: https://doi.org/10.5194/egusphere-2023-2165-RC2 -
AC2: 'Reply on RC2', Trude Eidhammer, 04 Jul 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2023-2165/egusphere-2023-2165-AC2-supplement.pdf
-
AC2: 'Reply on RC2', Trude Eidhammer, 04 Jul 2024
Peer review completion
Journal article(s) based on this preprint
Data sets
CESM2.2-CAM6 Perturbed Parameter Ensemble (PPE) Trude Eidhammer, Andrew Gettelman, and Katherine Thayer-Calder https://doi.org/10.26024/bzne-yf09
CERES_EBAF_Ed4.1_2001-2020 subset Trude Eidhammer and Andrew Gettelman https://zenodo.org/records/10426438
Viewed
HTML | XML | Total | BibTeX | EndNote | |
---|---|---|---|---|---|
805 | 385 | 37 | 1,227 | 28 | 28 |
- HTML: 805
- PDF: 385
- XML: 37
- Total: 1,227
- BibTeX: 28
- EndNote: 28
Viewed (geographical distribution)
Country | # | Views | % |
---|
Total: | 0 |
HTML: | 0 |
PDF: | 0 |
XML: | 0 |
- 1
Cited
Trude Eidhammer
Andrew Gettelman
Katherine Thayer-Calder
Duncan Watson-Parris
Gregory Elsaesser
Hugh Morrison
Marcus van Lier-Walqui
Ci Song
Daniel McCoy
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(15141 KB) - Metadata XML