the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Assessment of S2S ensemble extreme precipitation forecasts over Europe
Abstract. Heavy precipitation can lead to floods and landslides, resulting in widespread damage and significant casualties. Some of its impacts can be mitigated if reliable forecasts and warnings are available. Of particular interest is the subseasonal to seasonal (S2S) prediction timescale. The S2S prediction timescale has received increasing attention in the research community because of its importance for many sectors. However, very few forecast skill assessments of precipitation extremes in S2S forecast data have been conducted. The goal of this article is to introduce a new methodology to assess the skill of rare events, here extreme precipitation, in S2S forecasts. We verify extreme precipitation events over Europe in the S2S forecast model from the European Centre for Medium-Range Weather Forecasts. The verification is conducted against ERA5 reanalysis precipitation. Extreme precipitation is defined as daily precipitation accumulations exceeding the seasonal 95th percentile. In addition to the classical Brier score, we use a binary loss index to assess skill. The binary loss index is tailored to assess the skill of rare events. We analyse daily events locally and spatially aggregated, as well as 7-day extreme event counts. Results consistently show a higher skill in winter compared to summer. The regions showing the highest skill are Norway, Portugal and the south of the Alps. Skill increases when aggregating the extremes spatially or temporally. The verification methodology can be adapted and applied to other variables, e.g. temperature extremes or river discharge.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(5357 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(5357 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2022-1401', Anonymous Referee #1, 05 Jan 2023
Please see the attached pdf for my review.
- AC1: 'Reply on RC1', Pauline Rivoire, 16 May 2023
-
RC2: 'Comment on egusphere-2022-1401', Anonymous Referee #2, 23 Jan 2023
The authors aim to investigate the predictive ability of ECMWF S2S reforecast, focusing on intense precipitation events. S2S hindcast cover a range between medium-range and seasonal prediction. Currently this is one of the first work covering the skill of predicting extreme events at these long time ranges, is therefore interesting. They propose a relatively new index in measuring forecast accuracy (Binary Loss Index) although probably the same consideration would have been emerged using the Critical Success Index, the latter widely used in the meteorological community.
While the index computations and results are well presented, it is also true that discussion is weak in terms of meteorological implications. A part the obvious dependence of seasonality, with convective precipitation leading to less skill during summer, and spatial and temporal aggregation, would have been nice to investigate with greater detail the spatial variability of the lead time. I found the lead time definition useful and interesting, not as a number per se, as you correctly comment it depends on the level of event detection you want to achieve and it is user dependent, but as an index to investigate predictability and its dependency on other factors. Taking two regions of example, would have been interesting to composite days with very long time day, and events with a shorter long time day and show the differences in some upper level variable to infer the role of the precursors dynamical evolution leading to the precipitation extreme. In a way this goal was also mentioned in the introduction “Skill information is also useful to identify potential sources of predictability and windows of opportunity (i.e. intermittent time periods with higher skill Mariotti et al. (2020))”. The long time day could be used to detect those windows of opportunity; in which conditions they occur and for which regions are stronger. In that respect I think some elements in the paper could be inserted.
I have mixed feelings on the final judgment of the work. Since the title is Assessment of S2S ensemble extreme precipitation forecasts over Europe, I was expecting a more in depth discussion on practical predictability limit of precipitation extremes. For this reason I finally opted for major revision because I think the material is insufficient for this topic. But if you just wanted to present a new method to score extreme forecast, as you say in the abstract “ The goal of this article is to introduce a new methodology to assess the skill of rare events”, then the material could be sufficient but text and title needs to be restructured to put the accent on method. In the latter case a more in depth comparisons with results obtained with other scores is also needed.
Minor comments:
line 41 reference WCS(2021) looks weird. What is WCS ?
Citation: https://doi.org/10.5194/egusphere-2022-1401-RC2 - AC2: 'Reply on RC2', Pauline Rivoire, 16 May 2023
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2022-1401', Anonymous Referee #1, 05 Jan 2023
Please see the attached pdf for my review.
- AC1: 'Reply on RC1', Pauline Rivoire, 16 May 2023
-
RC2: 'Comment on egusphere-2022-1401', Anonymous Referee #2, 23 Jan 2023
The authors aim to investigate the predictive ability of ECMWF S2S reforecast, focusing on intense precipitation events. S2S hindcast cover a range between medium-range and seasonal prediction. Currently this is one of the first work covering the skill of predicting extreme events at these long time ranges, is therefore interesting. They propose a relatively new index in measuring forecast accuracy (Binary Loss Index) although probably the same consideration would have been emerged using the Critical Success Index, the latter widely used in the meteorological community.
While the index computations and results are well presented, it is also true that discussion is weak in terms of meteorological implications. A part the obvious dependence of seasonality, with convective precipitation leading to less skill during summer, and spatial and temporal aggregation, would have been nice to investigate with greater detail the spatial variability of the lead time. I found the lead time definition useful and interesting, not as a number per se, as you correctly comment it depends on the level of event detection you want to achieve and it is user dependent, but as an index to investigate predictability and its dependency on other factors. Taking two regions of example, would have been interesting to composite days with very long time day, and events with a shorter long time day and show the differences in some upper level variable to infer the role of the precursors dynamical evolution leading to the precipitation extreme. In a way this goal was also mentioned in the introduction “Skill information is also useful to identify potential sources of predictability and windows of opportunity (i.e. intermittent time periods with higher skill Mariotti et al. (2020))”. The long time day could be used to detect those windows of opportunity; in which conditions they occur and for which regions are stronger. In that respect I think some elements in the paper could be inserted.
I have mixed feelings on the final judgment of the work. Since the title is Assessment of S2S ensemble extreme precipitation forecasts over Europe, I was expecting a more in depth discussion on practical predictability limit of precipitation extremes. For this reason I finally opted for major revision because I think the material is insufficient for this topic. But if you just wanted to present a new method to score extreme forecast, as you say in the abstract “ The goal of this article is to introduce a new methodology to assess the skill of rare events”, then the material could be sufficient but text and title needs to be restructured to put the accent on method. In the latter case a more in depth comparisons with results obtained with other scores is also needed.
Minor comments:
line 41 reference WCS(2021) looks weird. What is WCS ?
Citation: https://doi.org/10.5194/egusphere-2022-1401-RC2 - AC2: 'Reply on RC2', Pauline Rivoire, 16 May 2023
Peer review completion
Journal article(s) based on this preprint
Viewed
HTML | XML | Total | BibTeX | EndNote | |
---|---|---|---|---|---|
371 | 166 | 22 | 559 | 13 | 8 |
- HTML: 371
- PDF: 166
- XML: 22
- Total: 559
- BibTeX: 13
- EndNote: 8
Viewed (geographical distribution)
Country | # | Views | % |
---|
Total: | 0 |
HTML: | 0 |
PDF: | 0 |
XML: | 0 |
- 1
Cited
1 citations as recorded by crossref.
Olivia Martius
Philippe Naveau
Alexandre Tuel
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(5357 KB) - Metadata XML