the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Analysis of model error in forecast errors of Extended Atmospheric Lorenz' 05 Systems and the ECMWF system
Hynek Bednář
Holger Kantz
Abstract. The forecast error growth as a function of lead time of atmospheric phenomena is caused by initial and model errors. When studying the initial error growth, it turns out that small scale phenomena, which contribute little to the forecast product, significantly affect the ability to predict this. The question under investigation is whether omitting these atmospheric phenomena will improve the predictability of the resulting value. The topic is studied in the extended Lorenz (2005) system. This system shows that omitting small spatiotemporal scales will reduce predictability more than modeling it. Generally, a system with model error (omitting phenomena) will not improve predictability. A theory explaining and describing this behavior is developed, with the difference between systems (model error) produced at each time step seen as the error of the initial conditions. The resulting model error is then defined as the sum of the increments of the time evolution of the initial conditions so defined. The theory is compared to the fit parameters that define the model error in certain approximations of the average forecast error growth. Parameters are interpreted in this context, and the hypotheses are used to estimate the errors described in the theory. It is proposed how to distinguish increments to prediction error growth from small spatiotemporal-scales phenomena and model error. Results are presented for the error growth of the ECMWF system, where a 40 % reduction in model error between 1987 and 2011 is calculated based on the developed theory, while over the same time, the instability of the system with respect to initial condition errors has grown.
- Preprint
(2444 KB) - Metadata XML
- BibTeX
- EndNote
Hynek Bednář and Holger Kantz
Status: open (extended)
-
RC1: 'Comment on egusphere-2023-1464', Anonymous Referee #1, 04 Sep 2023
reply
The authors consider the question of whether omitting small spatiotemporal scales atmospheric phenomena will improve the predictability of the resulting value, using the Lorenz (2005) systems denote by L05-1, L05-2, L05-3. This system shows that omitting small spatiotemporal scales will reduce predictability more than modeling it. The authors claimed that initial errors of magnitudes that are comparable to real weather forecasts do not play a dominant role in the L05 systems, while model errors do. Moreover, they present an explanation of the observed error growth in terms of an time-averaged model error called the drift.
The authors showed that for the multi-scale systems L05-2 and L05-3, the initial error growth can be well described by the power law or the extended power law, while a simple exponential growth or the quadratic hypothesis with model error are less appropriate. We showed that in the L05 and ECMWF data the error growth in the early stage grows faster than the approximation of the whole curve, probably due to the presence of only smaller spatiotemporal scales in this part.
The paper seem to be doing a good job recalling the basic notions of initial error growth and model error growth. The L05-3 system was introduced by the authors in an earlier paper to extend the 2-scale, L05-2 system to three scales. The parameters of these systems are set so that all scales behave chaotically. Though it is not totally clear how robust the results are if the parameters are perturbed. The explanation of the initial decline and subsequent growth of the rate of model error growth by the notion of ``drift" is a nice attempt, though it is not totally clear if this is special for the L-05 systems.
Overall, I think the results are worth publishing and the efforts is a nice demonstration of illustrating the effects of different scale on error growth.
I have only a few comments for improvement of the paper:
- In the abstract, where is the claim "Generally, a
system with model error (omitting phenomena) will not improve predictability." supported in the maintext? How general is it? This seems to be a very strong statement. If not, I suggest weaken this statement.- Although it maybe natural, it would be good to give a sentence of explanation about why choosing L05-2 and L05-3 as "reality"
- Is "this" in line 9 ``initial error growth"? Perhaps good to be more specific.
- How about adding references to the relevant figures after "as we will see in numerical simulations" in line 270?
- Would you explain why geometric mean is used rather than the usual arithmetic mean in model error growth and drift terms in (11), (14), (15), (18) ?
- The drift d(tau) at the beginning of line 271 is not defined yet, It does not seems to be the drift VECTOR in line 269. Please clarify.
- Please be consistent in terminology. For example, is ``the drift D(tau)" on Page 279 the same as d(tau) in line 271? Is it the same as the "the averaged drift D" in line 283?
- Perhaps include a table summarizing the heavy notation involved.
- The reference list will look better it it was itemized.
- I am not totally convinced (or understand) that the notion of the "drift" introduced really explain the model error growth as claimed. It seems that, taking time-average without an absolute value is similar to looking at the original system, when the system is ergodic. Perhaps the authors can explain more on what ``explain" means other than showing another summary statistics of the system.
Citation: https://doi.org/10.5194/egusphere-2023-1464-RC1 -
AC1: 'Reply on RC1', Hynek Bednar, 19 Sep 2023
reply
We are grateful to the referee for devoting their time to our manuscript. The valuable comments and suggestions will help us to improve the paper.
We will here respond to comments made:
The parameters of these systems are set so that all scales behave chaotically. Though it is not totally clear how robust the results are if the parameters are perturbed.
As long as parameters are such that all scales are chaotic, we do not expect any qualitative changes with respect to the studied scenario. We tried to ensure the robustness of the results by considering two cases of "reality" (L05-2 and L05-3 systems). Furthermore, we tested as "reality" the L05-1 system with 360 variables and as "model" the L05-1 system with 180 and 90 variables. The results are consistent with the presented results. We are aware of the need to test the results on "real" systems.The explanation of the initial decline and subsequent growth of the rate of model error growth by the notion of ``drift" is a nice attempt, though it is not totally clear if this is special for the L-05 systems.
"Drift" was used by Orrell (2002) to explain the initial decline and subsequent growth of the rate of model error growth for the ECMWF system (500 hPa, Northern hemisphere for 10 d in October 1999 and total energy globally over a 15 d period in December 2000) . Therefore, the results do not appear to apply only to the L05 system. We have further confirmed the behavior resulting from "drift" on the ECMWF system data in Section 5.In the abstract, where is the claim "Generally, a system with model error (omitting phenomena) will not improve predictability." supported in the maintext? How general is it? This seems to be a very strong statement. If not, I suggest weaken this statement.
We have replaced "Generally" with "In other words" (Lines 11 – 12).Although it maybe natural, it would be good to give a sentence of explanation about why choosing L05-2 and L05-3 as "reality"
A full explanation of why L05-2 and L05-3 systems are selected as the reality is given in Section 2.2. In addition, we have added a sentence to the introduction on lines 128 – 129 ("The omitted scale is the small scale for the L05-2 system and the small and medium scale for the L05-3 system. "). Information on why we do not use the L05-2 system as model and L05-3 system as reality is given on lines 614-615.Is "this" in line 9 ``initial error growth? Perhaps good to be more specific.
The word "product" was added to line 9.How about adding references to the relevant figures after "as we will see in numerical simulations" in line 270?
Reference was added.Would you explain why geometric mean is used rather than the usual arithmetic mean in model error growth and drift terms in (11), (14), (15), (18) ?
We added an explanation on line 200: "The geometric mean is chosen because of its suitability for comparison with growth governed by the largest Lyapunov exponent. For further information, see Bednar et al. (2014) or Ding and Li (2011). "The drift d(tau) at the beginning of line 271 is not defined yet, It does not seems to be the drift VECTOR in line 269. Please clarify.
d(tau) was changed to the absolute value of drift d(tau).Please be consistent in terminology. For example, is ``the drift D(tau)" on Page 279 the same as d(tau) in line 271? Is it the same as the "the averaged drift D" in line 283?
We improved consistency in terminology. We related D(t) to eq. (18) and d(tau) to eq. (17).Perhaps include a table summarizing the heavy notation involved.
We itemized the numbers of the equations. We hope this will help readability.The reference list will look better it it was itemized.
We itemized the reference list.I am not totally convinced (or understand) that the notion of the "drift" introduced really explain the model error growth as claimed. It seems that, taking time-average without an absolute value is similar to looking at the original system, when the system is ergodic. Perhaps the authors can explain more on what ``explain" means other than showing another summary statistics of the system.
The difference is that it is the summation of vectors created from the difference in time evolution of different systems (but with the same initial conditions) after one time step. The model errors at successive time steps as vectors are not strongly correlated, and that therefore accumulating their absolute values is very different from accumulating them as vectors, where the absolute values sum will grow much faster than the vector valued sum, and that this slower error growth now gives a better explanation of the deviation of the trajectories.References:
Bednář, H., Raidl, A., and Mikšovský, J.: Initial Error Growth and Predictability of Chaotic Low-dimensional Atmospheric Model, IJAC, 11, 256–264, https://doi.org/10.1007/s11633-014-0788-3 2014.
Ding, R., Li, J.: Comparisons of two ensemble mean methods in measuring the average error growth and the predictability, Acta Meteorol Sin, 25, 395–404, https://doi.org/10.1007/s13351-011-0401-4, 2011.
Orrell, D.: Role of the metric in forecast error growth: how chaotic is the weather?, Tellus, 54, 350-362, https://doi.org/10.1034/j.1600-0870.2002.01389.x, 2002.Citation: https://doi.org/10.5194/egusphere-2023-1464-AC1
-
AC1: 'Reply on RC1', Hynek Bednar, 19 Sep 2023
reply
Hynek Bednář and Holger Kantz
Data sets
Analysis of model error in forecast errors of Extended Atmospheric Lorenz' 05 Systems and the ECMWF system Hynek Bednář http://www.doi.org/10.17605/OSF.IO/2EWXB
Model code and software
Analysis of model error in forecast errors of Extended Atmospheric Lorenz' 05 Systems and the ECMWF system Hynek Bednář http://www.doi.org/10.17605/OSF.IO/2EWXB
Hynek Bednář and Holger Kantz
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 125 | 37 | 9 | 171 | 3 | 3 |
- HTML: 125
- PDF: 37
- XML: 9
- Total: 171
- BibTeX: 3
- EndNote: 3
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1