the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 25 Sep 2023
Towards a process-oriented understanding of the impact of stochastic perturbations on the model climate
Abstract. Stochastic parametrisation techniques have been used by operational weather centres for decades to produce ensemble forecasts and to represent uncertainties of the forecast model. Their use has been demonstrated to be highly beneficial, as it increases the reliability of the forecasting system and reduces systematic biases. Despite the random nature of the perturbation techniques, the response of the model can be nonlinear and the mean state of the model can change. In this study, we attempt to provide a process-based understanding how stochastic model perturbations affect the model climate. Previous work has revealed sensitivities of the occurrence of diabatically driven, rapidly ascending air streams to the stochastically perturbed parametrisation tendencies (SPPT) scheme. Such strongly ascending air streams are linked to different weather phenomena, such as precipitation and upper-tropospheric ridge building in the midlatitudes, which raises the question whether these processes are also influenced by stochastic perturbations.
First, we analyse if rapidly ascending air streams also show sensitivities to a different perturbation technique - the stochastically perturbed parametrisations (SPP) scheme, which directly represents parameter uncertainty in parametrisations and has recently been developed at the European Centre for Medium-Range Weather Forecasts (ECMWF). By running a set of sensitivity experiments with the Integrated Forecasting System (IFS) and by employing a Lagrangian detection of ascending air streams, we show that SPP results in a systematic increase of the occurrence of ascending air parcel trajectories compared to unperturbed simulations. This behaviour is very similar to that of SPPT, albeit some regional differences are apparent. We further show that the one-sided response to the stochastic forcing cannot be attributed to a single process (e.g. convection parametrisation), but rather that perturbations to different parametrisations have similar effects.
Thereafter, we link the frequency changes of ascending air streams to closely related weather phenomena. Whereas the signal of increased ascending motion is directly transmitted to global precipitation sums for all analysed schemes, changes to the amplitude of the upper-level Rossby wave pattern are more subtle. In agreement with the trajectory analysis, both SPPT and SPP increase the waviness of the upper-level flow and thereby reduce a systematic bias of the model, even though the order of magnitude is small.
Our study presents a coherent process chain that enables to understand how stochastic perturbations systematically affect the model climate. We argue that weather systems which are characterised by threshold behaviour on the one hand and that serve as a dynamical hinge between spatial scales on the other hand can convert zero-mean perturbations into an asymmetric response and project it onto larger scales.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(2689 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2689 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2023-1938', Anonymous Referee #1, 28 Nov 2023
The paper presents a study on how model error representation in the operational NWP forecast model of ECMWF can affect the model climate state. Here, stochastic perturbations to either the net physical tendencies or physical parameters for the parametrisation schemes are used to account for errors in the unresolved physical processes of the model. Their impact on the occurrence of diabatically driven, rapid ascending air streams – using trajectories – is analysed, resulting in systematically more frequent situations with such rapidly ascending air flows compared to unperturbed simulations without model error representations. The two considered stochastic schemes produce broadly similar results. Interestingly, the one-sided response to the stochastic forcing cannot be attributed to a single parametrized process (convection). It was further demonstrated how these systematic effects are directly linked to global precipitation statistics and to the amplitude of upper-level Rossby wave patterns. It was found that both stochastic schemes increase the waviness of the upper-level flow and thereby reduce the systematic bias of the model, even though the magnitude of the effect is small.
I really enjoyed reading this paper and think it is great addition to the existing literature on the effects of stochastic physical perturbations for model error contributions in numerical models. In particular, the process-based approach to understand the impact of the perturbations from the latent heat release along the ascending air streams to vertical velocity and precipitation and subsequently on the large-scale Rossby waves (amplitudes) is a very welcome advance over the often more statistical-in-nature studies that were carried out in the past. The experiments are well motivated, the results are both very interesting and presented in a clear and compelling way. I congratulate the authors on a great paper.
I only have a few minor suggestions and would certainly suggest publication of this study.
- Line 44: it might be worth to emphasise here that the SPPT scheme applies multiplicative random noise with certain spatial and temporal autocorrelation scales.
- Line 125: specify the year for which the experiments were run
- Section 3.1: discuss how different the various experiments perform with regards to the verification (ANA), mention in the discussion which of the differences are significant (the confidence intervals are not mentioned in the text even though they are plotted), in particular for the non-significant differences
- Same section and Fig 3: it would be helpful to briefly offer an interpretation of the heating rates. Maybe label the x-axis in the figure as heating rates in K/6h.
- Section 3.2.1: Does the range of negative differences in Fig 5 coincide with the range of the drizzle overestimation problem in many NWP forecast models?
- Fig 6: could confidence intervals be added, similar to Fig 7?
Citation: https://doi.org/10.5194/egusphere-2023-1938-RC1 -
RC2: 'Comment on egusphere-2023-1938', Anonymous Referee #2, 22 Jan 2024
This manuscript explores how stochastic model uncertainty perturbations impact rapidly ascending airstreams, also precipitation patterns and upper-level flow, providing a convincing process-level description of mean state changes that can result from the stochastic schemes. This work builds on previous work by the authors, which explored the impact of SPPT perturbations on WCBs.
The work is well designed and conducted. The presentation is very good and it was an enjoyable read. I think the study will make a valuable contribution to the literature. I have some suggestions for minor improvements – some specific and some more broad. None require much additional work, but they would improve the quality of the presentation.
One general criticism (detailed examples are described below): there are a number of instances in the manuscript where the authors make statements that imply greater breadth in the work than is presented or appear to overstate the size of differences in the experiment results. The work is very interesting, well-designed and well presented. There is no need to overstate the claims. Indeed, exposing the limitations highlights the areas that would benefit from further studies. I would ask the authors to be careful to present the work with complete accuracy – rely on the quality of the work to expose its merits and not consciously overstate them.
Detailed comments:
Abstract (& throughout): results are described as showing that “perturbations to *different parameterisations* have similar effects”. This a broad statement, which I find misleading. It implies more than what is shown in the paper. The SPP experiments demonstrate the impact of perturbations to the convection parameters alone (SPP-CONV-ONLY); and from perturbations to all other parameters (SPP-CONV-OFF). There are several parametrisations represented by the *CONV-OFF experiment, which could be interesting to explore, each in isolation (for a future study). I propose the authors take a less broad tone in describing the extent of the SPP exploration in this manuscript.
Related (e.g. line 194): the authors claim to analyse “other model uncertainty schemes [to SPPT]”. Again, I find this misleading. The study explores one other MU scheme (SPP) but in several configurations. A more accurate description would be that “other model uncertainty *representations* [have been analysed]”.
Line 24: remove “order of” – simply “the magnitude is small”
Lines 55-73: to add to the discussion, a recent paper demonstrates that SPPT perturbations applied to an active MJO region can be used to explore and understand the pathways of error growth from the tropics to the extra-tropics. Straus et al. (2023), https://doi.org/10.5194/wcd-4-1001-2023
Figure 1: the colour choice could be improved – the red and green can be difficult to distinguish. It is difficult to distinguish the blue and back lines (though the meaning is clear). In the shaded areas, there appear to be some red marks north of the equivalent latitude and some green marks to the south. According to the definition of troughs/ridges, this shouldn’t be possible – is it an error in the plotting?
Figure 2 and discussion (lines 195-212, also line 442): the differences between experiments do not all appear to be statistically significant: for the n. hem extra-tropics and the n. Atlantic, it is not obvious that there is any statistical significance in the differences between any of the experiments or the analysis. Unless I miss something, I would certainly refrain from making claims of differences between the SPP* experiments. Differences between others are perhaps “indicated”? For the tropics and globally, DET and IC-ONLY appear to be significantly different from the others; but, the error bars for the SPPT and SPP* experiments encompass the median of each of the others. If the differences are known to be statistically significant, please make that clear. If they are (known to be) not, please don’t overstate the differences.
Figure 3 and discussion (lines 220-230): further to the comment on Figure 2, the ratios of trajectory counts for the “extra-tropical regime” are very similar to each other. Without significance testing, I would be cautious about claiming (or believing there to be) any differences between the 4 experiments. For heating >40K, the differences do look clear and perhaps can be used to justify comments about differences between SPP, SPP-CONV-ONLY and SPP-CONV-OFF. Likewise, the comment (line 229) about SPPT for the smallest heating rates, given the small number of trajectories, I wonder whether the trajectory count ratio is really statistically different to 1.0?
Figure 3 caption: mentions an experiment “STOCDP” and a lightblue line that is not present in the figure.
Line 240: the inequalities are incorrectly expressed: the maximum for SPPT for slow ascents occurs for -0.2 < \omega < -0.05, and similarly for SPP.
Line 242: the omega range values are quoted the wrong way around (and incorrectly) for SPP and SPPT (according the figure): SPPT has a minimum for -0.4 < \omega < -0.2, and similarly for SPP.
Line 247: again, it is difficult to believe by eye from Figure 4 that the differences in the experiment lines for large +ve \omega demonstrate real differences, without some indication of significance testing.
Line 252: “balanced”: have you confirmed that the increased upward and downward mass fluxes generated by the stochastic perturbations do indeed balance?
Line 258: “number of grid points [with what?] is decreased”
Line 263-266: to be clear: the “uni-directional response” being that the perturbations tend to result in more grid-points with non-zero vertical motion? Could the authors spell this out for the reader in the text.
Line 282, missing word: “uncertainty schemes *on* two such phenomena”
Line 287 & 290 & 320 (+ elsewhere?): not the “unperturbed experiment” but the “unperturbed *physics* experiment” or simply “IC-ONLY” (which includes initial perturbations)
Line 268: add a word for clarity: “increased *occurrence* frequencies” (to avoid confusion with precipitation frequencies)
Line 294, missing word: “goes along *with* and might…”
Figure 5: it is not easy to read from the image, but is there something interesting happening to grid-points with zero precip? Would it also be informative (even possible?) to indicate the number of grid-points (in IC-ONLY) for each precip rate (and omega in Fig 4)? Similar to what has been done in Figure 3. To give an impression of how widespread any changes in the rates are across the model.
Line 300, incorrect internal reference: should “Chapter 2” be “Figure 1” or “section 2.4.2”?
Lines 355-378: the differences between IC-ONLY and the SPP* experiments are small and by eye (Figure 7), do not suggest they are significant. The authors make this point at the end of the paragraph and the section, but only after the reader has read many lines describing minor differences. I propose highlighting the likely lack of significance (and the upcoming section to enhance the ability for statistical testing) earlier in the paragraph and not overstate the differences displayed in Figure 7.
Line 387, typo: should be “3,200’ (not ‘.’)
Figure 8: I wonder if placing all 3 seasons on the same vertical axis would enhance the impression of the larger signal for SON. It looks like the different seasons (in particular, DJF and MAM) would not overlay each other too much; and the relative size of the signals would be much clearer.
Lines 443-445: again, this reads as a more general statement on SPP perturbations to individual parametrisations than are shown by the results in this study. Have the authors tested if, for example, the model response is the same from perturbations to the boundary layer scheme (only) and those to the cloud schemes (only)? The study only demonstrates CONV-ONLY and CONV-OFF. More detailed testing can be for a future study, but refrain from making claims that are broader than the scope of this study.
Line 465: I couldn’t immediately identify where the “10-20%” figure was identified in the earlier results sections. The text references Figures 2 and 9, but it’s still not entirely obvious.
Line 478: “likely not of the same order of magnitude as the trajectory count” – I understand that the impact on the two components is likely to be different, but why would the differences likely be different *orders of magnitude*?
Citation: https://doi.org/10.5194/egusphere-2023-1938-RC2 -
AC1: 'Comment on egusphere-2023-1938', Moritz Deinhard, 04 Feb 2024
We would like to thank the reviewers for their time and effort in thoroughly reviewing our manuscript and for their valuable feedback on our study.
Based on the feedback of the reviewers, we will implement some of the suggested changes to the manuscript and will carefully work on the text. One aspect that was mentioned by the reviewers is that in some parts of the paper we would make too strong statements about differences between experiments and that we draw broader conclusions than we should based on the presented results. We take this feedback very seriously and will carefully review our statements and align them with the presented results. However, we never intended to overstate our results in any way and therefore politely reject this criticism. We rather realize that some apparent misunderstandings have arisen from some unclear statements in the original manuscript. For example, the intention of discussing small differences between the experiments SPP-CONV-OFF and SPP-CONF-ONLY were not meant to overstate the differences, but rather to show that they behave mostly similar, but not identical. In the re-submission, we will focus on avoiding such possible misunderstandings.
Apart from this, we will implement the proposed changes to the text and the feedback to the figures, and are hopeful to submit an improved manuscript.
Citation: https://doi.org/10.5194/egusphere-2023-1938-AC1
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2023-1938', Anonymous Referee #1, 28 Nov 2023
The paper presents a study on how model error representation in the operational NWP forecast model of ECMWF can affect the model climate state. Here, stochastic perturbations to either the net physical tendencies or physical parameters for the parametrisation schemes are used to account for errors in the unresolved physical processes of the model. Their impact on the occurrence of diabatically driven, rapid ascending air streams – using trajectories – is analysed, resulting in systematically more frequent situations with such rapidly ascending air flows compared to unperturbed simulations without model error representations. The two considered stochastic schemes produce broadly similar results. Interestingly, the one-sided response to the stochastic forcing cannot be attributed to a single parametrized process (convection). It was further demonstrated how these systematic effects are directly linked to global precipitation statistics and to the amplitude of upper-level Rossby wave patterns. It was found that both stochastic schemes increase the waviness of the upper-level flow and thereby reduce the systematic bias of the model, even though the magnitude of the effect is small.
I really enjoyed reading this paper and think it is great addition to the existing literature on the effects of stochastic physical perturbations for model error contributions in numerical models. In particular, the process-based approach to understand the impact of the perturbations from the latent heat release along the ascending air streams to vertical velocity and precipitation and subsequently on the large-scale Rossby waves (amplitudes) is a very welcome advance over the often more statistical-in-nature studies that were carried out in the past. The experiments are well motivated, the results are both very interesting and presented in a clear and compelling way. I congratulate the authors on a great paper.
I only have a few minor suggestions and would certainly suggest publication of this study.
- Line 44: it might be worth to emphasise here that the SPPT scheme applies multiplicative random noise with certain spatial and temporal autocorrelation scales.
- Line 125: specify the year for which the experiments were run
- Section 3.1: discuss how different the various experiments perform with regards to the verification (ANA), mention in the discussion which of the differences are significant (the confidence intervals are not mentioned in the text even though they are plotted), in particular for the non-significant differences
- Same section and Fig 3: it would be helpful to briefly offer an interpretation of the heating rates. Maybe label the x-axis in the figure as heating rates in K/6h.
- Section 3.2.1: Does the range of negative differences in Fig 5 coincide with the range of the drizzle overestimation problem in many NWP forecast models?
- Fig 6: could confidence intervals be added, similar to Fig 7?
Citation: https://doi.org/10.5194/egusphere-2023-1938-RC1 -
RC2: 'Comment on egusphere-2023-1938', Anonymous Referee #2, 22 Jan 2024
This manuscript explores how stochastic model uncertainty perturbations impact rapidly ascending airstreams, also precipitation patterns and upper-level flow, providing a convincing process-level description of mean state changes that can result from the stochastic schemes. This work builds on previous work by the authors, which explored the impact of SPPT perturbations on WCBs.
The work is well designed and conducted. The presentation is very good and it was an enjoyable read. I think the study will make a valuable contribution to the literature. I have some suggestions for minor improvements – some specific and some more broad. None require much additional work, but they would improve the quality of the presentation.
One general criticism (detailed examples are described below): there are a number of instances in the manuscript where the authors make statements that imply greater breadth in the work than is presented or appear to overstate the size of differences in the experiment results. The work is very interesting, well-designed and well presented. There is no need to overstate the claims. Indeed, exposing the limitations highlights the areas that would benefit from further studies. I would ask the authors to be careful to present the work with complete accuracy – rely on the quality of the work to expose its merits and not consciously overstate them.
Detailed comments:
Abstract (& throughout): results are described as showing that “perturbations to *different parameterisations* have similar effects”. This a broad statement, which I find misleading. It implies more than what is shown in the paper. The SPP experiments demonstrate the impact of perturbations to the convection parameters alone (SPP-CONV-ONLY); and from perturbations to all other parameters (SPP-CONV-OFF). There are several parametrisations represented by the *CONV-OFF experiment, which could be interesting to explore, each in isolation (for a future study). I propose the authors take a less broad tone in describing the extent of the SPP exploration in this manuscript.
Related (e.g. line 194): the authors claim to analyse “other model uncertainty schemes [to SPPT]”. Again, I find this misleading. The study explores one other MU scheme (SPP) but in several configurations. A more accurate description would be that “other model uncertainty *representations* [have been analysed]”.
Line 24: remove “order of” – simply “the magnitude is small”
Lines 55-73: to add to the discussion, a recent paper demonstrates that SPPT perturbations applied to an active MJO region can be used to explore and understand the pathways of error growth from the tropics to the extra-tropics. Straus et al. (2023), https://doi.org/10.5194/wcd-4-1001-2023
Figure 1: the colour choice could be improved – the red and green can be difficult to distinguish. It is difficult to distinguish the blue and back lines (though the meaning is clear). In the shaded areas, there appear to be some red marks north of the equivalent latitude and some green marks to the south. According to the definition of troughs/ridges, this shouldn’t be possible – is it an error in the plotting?
Figure 2 and discussion (lines 195-212, also line 442): the differences between experiments do not all appear to be statistically significant: for the n. hem extra-tropics and the n. Atlantic, it is not obvious that there is any statistical significance in the differences between any of the experiments or the analysis. Unless I miss something, I would certainly refrain from making claims of differences between the SPP* experiments. Differences between others are perhaps “indicated”? For the tropics and globally, DET and IC-ONLY appear to be significantly different from the others; but, the error bars for the SPPT and SPP* experiments encompass the median of each of the others. If the differences are known to be statistically significant, please make that clear. If they are (known to be) not, please don’t overstate the differences.
Figure 3 and discussion (lines 220-230): further to the comment on Figure 2, the ratios of trajectory counts for the “extra-tropical regime” are very similar to each other. Without significance testing, I would be cautious about claiming (or believing there to be) any differences between the 4 experiments. For heating >40K, the differences do look clear and perhaps can be used to justify comments about differences between SPP, SPP-CONV-ONLY and SPP-CONV-OFF. Likewise, the comment (line 229) about SPPT for the smallest heating rates, given the small number of trajectories, I wonder whether the trajectory count ratio is really statistically different to 1.0?
Figure 3 caption: mentions an experiment “STOCDP” and a lightblue line that is not present in the figure.
Line 240: the inequalities are incorrectly expressed: the maximum for SPPT for slow ascents occurs for -0.2 < \omega < -0.05, and similarly for SPP.
Line 242: the omega range values are quoted the wrong way around (and incorrectly) for SPP and SPPT (according the figure): SPPT has a minimum for -0.4 < \omega < -0.2, and similarly for SPP.
Line 247: again, it is difficult to believe by eye from Figure 4 that the differences in the experiment lines for large +ve \omega demonstrate real differences, without some indication of significance testing.
Line 252: “balanced”: have you confirmed that the increased upward and downward mass fluxes generated by the stochastic perturbations do indeed balance?
Line 258: “number of grid points [with what?] is decreased”
Line 263-266: to be clear: the “uni-directional response” being that the perturbations tend to result in more grid-points with non-zero vertical motion? Could the authors spell this out for the reader in the text.
Line 282, missing word: “uncertainty schemes *on* two such phenomena”
Line 287 & 290 & 320 (+ elsewhere?): not the “unperturbed experiment” but the “unperturbed *physics* experiment” or simply “IC-ONLY” (which includes initial perturbations)
Line 268: add a word for clarity: “increased *occurrence* frequencies” (to avoid confusion with precipitation frequencies)
Line 294, missing word: “goes along *with* and might…”
Figure 5: it is not easy to read from the image, but is there something interesting happening to grid-points with zero precip? Would it also be informative (even possible?) to indicate the number of grid-points (in IC-ONLY) for each precip rate (and omega in Fig 4)? Similar to what has been done in Figure 3. To give an impression of how widespread any changes in the rates are across the model.
Line 300, incorrect internal reference: should “Chapter 2” be “Figure 1” or “section 2.4.2”?
Lines 355-378: the differences between IC-ONLY and the SPP* experiments are small and by eye (Figure 7), do not suggest they are significant. The authors make this point at the end of the paragraph and the section, but only after the reader has read many lines describing minor differences. I propose highlighting the likely lack of significance (and the upcoming section to enhance the ability for statistical testing) earlier in the paragraph and not overstate the differences displayed in Figure 7.
Line 387, typo: should be “3,200’ (not ‘.’)
Figure 8: I wonder if placing all 3 seasons on the same vertical axis would enhance the impression of the larger signal for SON. It looks like the different seasons (in particular, DJF and MAM) would not overlay each other too much; and the relative size of the signals would be much clearer.
Lines 443-445: again, this reads as a more general statement on SPP perturbations to individual parametrisations than are shown by the results in this study. Have the authors tested if, for example, the model response is the same from perturbations to the boundary layer scheme (only) and those to the cloud schemes (only)? The study only demonstrates CONV-ONLY and CONV-OFF. More detailed testing can be for a future study, but refrain from making claims that are broader than the scope of this study.
Line 465: I couldn’t immediately identify where the “10-20%” figure was identified in the earlier results sections. The text references Figures 2 and 9, but it’s still not entirely obvious.
Line 478: “likely not of the same order of magnitude as the trajectory count” – I understand that the impact on the two components is likely to be different, but why would the differences likely be different *orders of magnitude*?
Citation: https://doi.org/10.5194/egusphere-2023-1938-RC2 -
AC1: 'Comment on egusphere-2023-1938', Moritz Deinhard, 04 Feb 2024
We would like to thank the reviewers for their time and effort in thoroughly reviewing our manuscript and for their valuable feedback on our study.
Based on the feedback of the reviewers, we will implement some of the suggested changes to the manuscript and will carefully work on the text. One aspect that was mentioned by the reviewers is that in some parts of the paper we would make too strong statements about differences between experiments and that we draw broader conclusions than we should based on the presented results. We take this feedback very seriously and will carefully review our statements and align them with the presented results. However, we never intended to overstate our results in any way and therefore politely reject this criticism. We rather realize that some apparent misunderstandings have arisen from some unclear statements in the original manuscript. For example, the intention of discussing small differences between the experiments SPP-CONV-OFF and SPP-CONF-ONLY were not meant to overstate the differences, but rather to show that they behave mostly similar, but not identical. In the re-submission, we will focus on avoiding such possible misunderstandings.
Apart from this, we will implement the proposed changes to the text and the feedback to the figures, and are hopeful to submit an improved manuscript.
Citation: https://doi.org/10.5194/egusphere-2023-1938-AC1
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 355 | 117 | 19 | 491 | 14 | 13 |
- HTML: 355
- PDF: 117
- XML: 19
- Total: 491
- BibTeX: 14
- EndNote: 13
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Moritz Deinhard
Christian M. Grams
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2689 KB) - Metadata XML