the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 06 Apr 2022
An improved near real-time precipitation retrieval for Brazil
Simon Pfreundschuh
Ingrid Ingemarsson
Patrick Eriksson
Daniel Alejandro Vila
Alan James P. Calheiros
Abstract. Observations from geostationary satellites offer the unique ability to provide spatially continuous coverage at continental scales with high spatial and temporal resolution. Because of this, they are commonly used to complement ground-based measurements of precipitation, whose coverage is often more limited.
We present a novel, neural-network-based, near real-time precipitation retrieval for Brazil based on visible and infrared (VIS/IR) observations from the Advanced Baseline Imager on the Geostationary Operational Environmental Satellite 16. The retrieval, which employs a convolutional neural network to perform Bayesian retrievals of precipitation, was developed with the aims of (1) leveraging the full potential of latest-generation geostationary observations and (2) providing probabilistic precipitation estimates with well-calibrated uncertainties. The retrieval is trained using co-locations with combined radar and radiometer retrievals from the Global Precipitation Measurement (GPM) Core Observatory. Its accuracy is assessed using one month of gauge measurements and compared to the precipitation retrieval that is currently in operational use at the Brazilian Institute for Space Research as well as two state-of-the-art global precipitation products: The Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks Cloud Classification System and the Integrated Multi-Satellite Retrievals for GPM (IMERG). Even in its most basic configuration, the accuracy of the proposed retrieval is similar to that of IMERG, which merges retrievals from VIS/IR and microwave observations with gauge measurements. In its most advanced configuration, the retrieval reduces the mean absolute error for hourly accumulations by 22 % compared the currently operational retrieval, by 50 % for the MSE and increases the correlation by 400 %. Compared to IMERG, the improvements correspond to 15 %, 15 % and 39 %, respectively. Furthermore, we show that the probabilistic retrieval is well calibrated against gauge measurements when differences in a priori distributions are accounted for.
In addition to potential improvements in near real time precipitation estimation over Brazil, our findings highlight the potential of specialized data driven retrievals that are made possible through advances in geostationary sensor technology, the availability of high-quality reference measurements from the GPM mission and modern machine learning techniques. Furthermore, our results show the potential of probabilistic precipitation retrievals to better characterize the observed precipitation and provide more trustworthy retrieval results.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(9103 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(9103 KB) - BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Simon Pfreundschuh et al.
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2022-78', Anonymous Referee #1, 07 May 2022
An improved near real-time precipitation retrieval for Brazil
By Pfreundschuh et al.,
The paper presents a convolutional neural network architecture for IR precipitation retrieval over Brazil. The training data are from IR and GPM combined retrievals. The framework is extended such that it can provide uncertainty of the retrievals. The estimates are compared with ground-based gauge data to validate the retrievals. The paper is well written and is of high-quality. I have the following comments.
Major comments:
- Validation only with a month of gauge data is not sufficient for clamming those improved results in the abstract. Seasonal to annual validation results are needed to make those claims.
- A single storm retrieval is missing. It is imperative to show the output of the algorithm in retrieval of a single or multiple storms and compare the results with the combined GPM retrievals as a reference. One retrieval snapshot speaks very clearly about the skill of the algorithm in reconstructing the training data and retrieve spatial structure of precipitation.
- Error metrics are only represented cumulatively. The expectation is that paper presents the quality of retrievals for an individual storm in terms of detection accuracy (e.g., probability of detection, miss) and then focuses on estimation quality metrics at different time scales from a storm scale to monthly and seasonal.
- This needs to be clarified whether the training data were only over Brazil or not. If this is the case, then the provided improved statics are not of surprise. This issue needs to be stated in the abstract.
- The way the paper explains the Bayesian retrieval is confusing. First, what is the prior distribution? Just obtaining uncertainty of estimates does not mean that the approach is Bayesian, and we can call the distribution a posterior. We can quantify uncertainty in a frequentists sense. It seems that the approach counts the number of retrievals associated with Tbs within bins. Then the bin with maximum is labeled. The problem is then defined as classification problem and the output of the softmax function is considered as the posterior distribution of the retrievals. Even though, I found the approach creative, I am not convinced that it is a Bayesian approach.
- It is claimed that spatially aware CNNs provide more accurate retrievals than pixel-level DNNs. The reason is not discussed, and no evidence is provided.
- In equation 2, when the prior probability approaches to a small number, the likelihood ratio can be extremely large. The correction numbers in Fig. 4 are too large. Please explain why such a large difference might exist in the retrievals that need such a large correction factor. For correcting probability distribution we can use a simple CDF matching!
- The resolution of IR is higher than microwave data. In this sense, you have redundant samples. How were those samples treated in the training?
- Explanation of the uncertainty quantification is too complex. Please consider simplifying the text and provide improved explanations.
Some minor comments
- Why both the second and third configurations are needed. They are just different in resolution.
- Line 160. Provide reasoning.
- Line 185”. The range is too wide! The training GPM combined precipitation can range from 0.1 to 200 mm/hr. Why 1000 mm/hr?
Citation: https://doi.org/10.5194/egusphere-2022-78-RC1 - AC1: 'Reply on RC1', Simon Pfreundschuh, 19 Aug 2022
- AC2: 'Reply on RC1', Simon Pfreundschuh, 19 Aug 2022
-
RC2: 'Comment on egusphere-2022-78', Anonymous Referee #2, 17 Jun 2022
General Comments: This is an interesting and generally well-written manuscript describing the use of a Convolutional Neural Network (CNN) architecture to retrieve rain rates from the GOES-16 satellite. This effort represents a very significant scientific contribution with results that are very encouraging when compared to other state-of-the-art algorithms. The only significant concern is that the manuscript as presently written assumes a significant prior understanding of these methodologies and thus may not be appropriate for a general audience. A few additional paragraphs of general background information on CNNs and the other techniques used in this manuscript would significantly improve it.
Specific Comments:
- Line 148: What is the purpose for the 4-km experiments given that the ABI has a native resolution of 2 km?
- Line 152: The availability of sunlight does not affect the other IR and WV bands, only the availability of VIS bands. Therefore, it does not justify the use of only a single IR channel. Please clarify the reasoning here.
- Lines 155-156: How are the values of the visible and near-IR bands treated by the CNN to differentiate daytime from nighttime scenes? Or is this something the CNN does without any intervention?
- Sections 3.3 and 3.4: The description of the CNN needs much more detail to be understood by readers who are not experts on CNNs.
- Lines 181-183: Please briefly define terms such as “cross-entropy loss”, “logits”, and “softmax activation” that would probably be unfamiliar to most readers.
- Line 165: Why is downsampling done for the 2-km retrievals rather than the 4-km retrievals? Shouldn’t it be the other way around?
- Line 167: Please provide references to support this assertion about the number of internal features relative to other architectures.
- Line 178: What in particular makes it easier to compute this sum on the binned PDF than on the quantiles? Please explain this more thoroughly.
- Line 186, 442: Please explain what “the degeneracy of (low) quantiles” means.
- Line 189: What is the rationale for creating outputs for 128 bins if only 14 quantiles are used?
- Line 193: What does “inference” mean in this context?
- Line 196: What does “posterior” mean in this context?
- Lines 203-204: Why specifically will assuming that the retrieval uncertainty is temporally independent cause the uncertainty to decay for consecutive identical observations?
- Lines 279-280: This is true, but it would be very scientifically interesting to see the relative degree of improvement during the day and night e.g., to quantify the value of the visible and near-IR data.
- Line 342: Why does assuming dependent retrieval errors lead to the uncertainties being overestimated?
- Lines 352-355: Please explain the precision-recall curves in Fig. 11 in more detail; they are difficult to understand from the explanation provided.
- Lines 361-362 and 473: How precisely does varying the probability threshold have a “calibrating effect” on the retrieval results?
- Lines 378-379: What probability threshold was tuned, and why was a FAR close to IMERG the criterion for doing so?
- Table 4: Why precisely does correcting when assuming independent errors actually degrade the POD, FAR, and CSI relative to the uncorrected version?
- Figure 13: The use of grayscale for the rain rates and colors for the errors makes the plots very hard to read. Would it be possible to instead plot the satellite rain rates in color and plot the corresponding gauge values using the same color scheme? Similar values would have very little contrast whereas large errors would produce sharp contrasts.
- Line 432 and Fig. 15: Please define more precisely what the 99th percentile of the distribution means. If each point in Fig. 15 is the 99th percentile of all of the rainfall values for a particular gauge location during the month of Dec. 2020, why are there so many values < 5 mm/h? Is it the dry season in some of these locations?
- Lines 435-436: Are there any specific assertions in the published literature that HYDRO and PERSIANN-CCS were both developed to correctly represent heavy precipitation at the presumed expense of skill for lighter precipitation?
Technical Comments:
- Line 39: For consistency, it might be better to cite Schmit et al. (2018) instead of Schmit et al. 2005) since the former is cited in lines 65 and 129.
- Line 46, 55, 93, 574-577: Scofield and Kuligowski (2003a) and (2003b) are the same paper.
- Line 54: Please cite Nguyen et al. (2020) here in reference to PERSIANN-PDIR.
- Line 64: Is “Hydronn” an acronym (e.g., Hydro-Neural Network) or does the name have a different meaning?
- Line 80: Replace “consists” with “consist” (“measurements” is plural).
- Lines 85, 86, 88: “Northwest” should not be capitalized unless it is a proper name.
- Line 86 Many readers may not know that “Amazonas” is the proper name for a state in Brazil, so “the Brazilian state of Amazonas” would be clearer.
- Line 88: Replace “manifest” with e.g., “is associated with”.
- Line 118: Replace “available first” with “available only”.
- Line 135: replace “criterion” with “approach”.
- Lines 133, 386, and elsewhere: please ensure that all dates in this manuscript match the format used in EGUsphere.
- Line 150: A better wording would be “a long time series of geostationary sensors”.
- Line 350: Insert “to” before “derive”.
- Line 354: Is “retrieved” meant rather than “predicted”?
- Line 354: “Pixel” should be plural.
- Line 362: Is worse detection accuracy than at 5 mm/h meant here?
- Figure 12 caption: add “at a rate of 5 mm/h” to the end of the caption for clarity.
- Line 387: “Floodings” should be singular or replaced with “floods”.
- Lines 387, 404, 520: Is this citation and reference formatted correctly?
- Line 388: Replace “were” with “was”.
- Line 394: Please indicate the location of Duque de Caxias in Fig. 13.
- Line 430: Replace “by” with “of”.
- Line 431: “Runoff” is a single word.
- Line 433: “Station” should be plural.
- Line 434: Replace “similar accuracy as” with “accuracy similar to”.
- Line 461: A more precise wording might be “correct for variations in the distribution of precipitation rates in the training data relative to comparable ground validation data.”
- Line 465: Replace “stronger” with “more strongly”.
- Line 473: Replace “small” with “low”.
- Line 475: Constant in time, space, or both?
- Line 485: Please define “GPM CO” in line 69 so the acronym is already defined.
- Lines 485-486: the latitude range of the GPM DPR is actually 65â°S to 65â°N when the instrument swath is accounted for.
- Line 489: This is the first time that the CNN is described as a “probabilistic regression” approach; this concept should be introduced earlier in the manuscript.
- Line 494: Delete the comma after “resolutions”.
- Line 587: “The Python Language Foundation” should be considered as starting with “P” since “The” is ignored when alphabetizing entries.
Citation: https://doi.org/10.5194/egusphere-2022-78-RC2 - AC3: 'Reply on RC2', Simon Pfreundschuh, 19 Aug 2022
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2022-78', Anonymous Referee #1, 07 May 2022
An improved near real-time precipitation retrieval for Brazil
By Pfreundschuh et al.,
The paper presents a convolutional neural network architecture for IR precipitation retrieval over Brazil. The training data are from IR and GPM combined retrievals. The framework is extended such that it can provide uncertainty of the retrievals. The estimates are compared with ground-based gauge data to validate the retrievals. The paper is well written and is of high-quality. I have the following comments.
Major comments:
- Validation only with a month of gauge data is not sufficient for clamming those improved results in the abstract. Seasonal to annual validation results are needed to make those claims.
- A single storm retrieval is missing. It is imperative to show the output of the algorithm in retrieval of a single or multiple storms and compare the results with the combined GPM retrievals as a reference. One retrieval snapshot speaks very clearly about the skill of the algorithm in reconstructing the training data and retrieve spatial structure of precipitation.
- Error metrics are only represented cumulatively. The expectation is that paper presents the quality of retrievals for an individual storm in terms of detection accuracy (e.g., probability of detection, miss) and then focuses on estimation quality metrics at different time scales from a storm scale to monthly and seasonal.
- This needs to be clarified whether the training data were only over Brazil or not. If this is the case, then the provided improved statics are not of surprise. This issue needs to be stated in the abstract.
- The way the paper explains the Bayesian retrieval is confusing. First, what is the prior distribution? Just obtaining uncertainty of estimates does not mean that the approach is Bayesian, and we can call the distribution a posterior. We can quantify uncertainty in a frequentists sense. It seems that the approach counts the number of retrievals associated with Tbs within bins. Then the bin with maximum is labeled. The problem is then defined as classification problem and the output of the softmax function is considered as the posterior distribution of the retrievals. Even though, I found the approach creative, I am not convinced that it is a Bayesian approach.
- It is claimed that spatially aware CNNs provide more accurate retrievals than pixel-level DNNs. The reason is not discussed, and no evidence is provided.
- In equation 2, when the prior probability approaches to a small number, the likelihood ratio can be extremely large. The correction numbers in Fig. 4 are too large. Please explain why such a large difference might exist in the retrievals that need such a large correction factor. For correcting probability distribution we can use a simple CDF matching!
- The resolution of IR is higher than microwave data. In this sense, you have redundant samples. How were those samples treated in the training?
- Explanation of the uncertainty quantification is too complex. Please consider simplifying the text and provide improved explanations.
Some minor comments
- Why both the second and third configurations are needed. They are just different in resolution.
- Line 160. Provide reasoning.
- Line 185”. The range is too wide! The training GPM combined precipitation can range from 0.1 to 200 mm/hr. Why 1000 mm/hr?
Citation: https://doi.org/10.5194/egusphere-2022-78-RC1 - AC1: 'Reply on RC1', Simon Pfreundschuh, 19 Aug 2022
- AC2: 'Reply on RC1', Simon Pfreundschuh, 19 Aug 2022
-
RC2: 'Comment on egusphere-2022-78', Anonymous Referee #2, 17 Jun 2022
General Comments: This is an interesting and generally well-written manuscript describing the use of a Convolutional Neural Network (CNN) architecture to retrieve rain rates from the GOES-16 satellite. This effort represents a very significant scientific contribution with results that are very encouraging when compared to other state-of-the-art algorithms. The only significant concern is that the manuscript as presently written assumes a significant prior understanding of these methodologies and thus may not be appropriate for a general audience. A few additional paragraphs of general background information on CNNs and the other techniques used in this manuscript would significantly improve it.
Specific Comments:
- Line 148: What is the purpose for the 4-km experiments given that the ABI has a native resolution of 2 km?
- Line 152: The availability of sunlight does not affect the other IR and WV bands, only the availability of VIS bands. Therefore, it does not justify the use of only a single IR channel. Please clarify the reasoning here.
- Lines 155-156: How are the values of the visible and near-IR bands treated by the CNN to differentiate daytime from nighttime scenes? Or is this something the CNN does without any intervention?
- Sections 3.3 and 3.4: The description of the CNN needs much more detail to be understood by readers who are not experts on CNNs.
- Lines 181-183: Please briefly define terms such as “cross-entropy loss”, “logits”, and “softmax activation” that would probably be unfamiliar to most readers.
- Line 165: Why is downsampling done for the 2-km retrievals rather than the 4-km retrievals? Shouldn’t it be the other way around?
- Line 167: Please provide references to support this assertion about the number of internal features relative to other architectures.
- Line 178: What in particular makes it easier to compute this sum on the binned PDF than on the quantiles? Please explain this more thoroughly.
- Line 186, 442: Please explain what “the degeneracy of (low) quantiles” means.
- Line 189: What is the rationale for creating outputs for 128 bins if only 14 quantiles are used?
- Line 193: What does “inference” mean in this context?
- Line 196: What does “posterior” mean in this context?
- Lines 203-204: Why specifically will assuming that the retrieval uncertainty is temporally independent cause the uncertainty to decay for consecutive identical observations?
- Lines 279-280: This is true, but it would be very scientifically interesting to see the relative degree of improvement during the day and night e.g., to quantify the value of the visible and near-IR data.
- Line 342: Why does assuming dependent retrieval errors lead to the uncertainties being overestimated?
- Lines 352-355: Please explain the precision-recall curves in Fig. 11 in more detail; they are difficult to understand from the explanation provided.
- Lines 361-362 and 473: How precisely does varying the probability threshold have a “calibrating effect” on the retrieval results?
- Lines 378-379: What probability threshold was tuned, and why was a FAR close to IMERG the criterion for doing so?
- Table 4: Why precisely does correcting when assuming independent errors actually degrade the POD, FAR, and CSI relative to the uncorrected version?
- Figure 13: The use of grayscale for the rain rates and colors for the errors makes the plots very hard to read. Would it be possible to instead plot the satellite rain rates in color and plot the corresponding gauge values using the same color scheme? Similar values would have very little contrast whereas large errors would produce sharp contrasts.
- Line 432 and Fig. 15: Please define more precisely what the 99th percentile of the distribution means. If each point in Fig. 15 is the 99th percentile of all of the rainfall values for a particular gauge location during the month of Dec. 2020, why are there so many values < 5 mm/h? Is it the dry season in some of these locations?
- Lines 435-436: Are there any specific assertions in the published literature that HYDRO and PERSIANN-CCS were both developed to correctly represent heavy precipitation at the presumed expense of skill for lighter precipitation?
Technical Comments:
- Line 39: For consistency, it might be better to cite Schmit et al. (2018) instead of Schmit et al. 2005) since the former is cited in lines 65 and 129.
- Line 46, 55, 93, 574-577: Scofield and Kuligowski (2003a) and (2003b) are the same paper.
- Line 54: Please cite Nguyen et al. (2020) here in reference to PERSIANN-PDIR.
- Line 64: Is “Hydronn” an acronym (e.g., Hydro-Neural Network) or does the name have a different meaning?
- Line 80: Replace “consists” with “consist” (“measurements” is plural).
- Lines 85, 86, 88: “Northwest” should not be capitalized unless it is a proper name.
- Line 86 Many readers may not know that “Amazonas” is the proper name for a state in Brazil, so “the Brazilian state of Amazonas” would be clearer.
- Line 88: Replace “manifest” with e.g., “is associated with”.
- Line 118: Replace “available first” with “available only”.
- Line 135: replace “criterion” with “approach”.
- Lines 133, 386, and elsewhere: please ensure that all dates in this manuscript match the format used in EGUsphere.
- Line 150: A better wording would be “a long time series of geostationary sensors”.
- Line 350: Insert “to” before “derive”.
- Line 354: Is “retrieved” meant rather than “predicted”?
- Line 354: “Pixel” should be plural.
- Line 362: Is worse detection accuracy than at 5 mm/h meant here?
- Figure 12 caption: add “at a rate of 5 mm/h” to the end of the caption for clarity.
- Line 387: “Floodings” should be singular or replaced with “floods”.
- Lines 387, 404, 520: Is this citation and reference formatted correctly?
- Line 388: Replace “were” with “was”.
- Line 394: Please indicate the location of Duque de Caxias in Fig. 13.
- Line 430: Replace “by” with “of”.
- Line 431: “Runoff” is a single word.
- Line 433: “Station” should be plural.
- Line 434: Replace “similar accuracy as” with “accuracy similar to”.
- Line 461: A more precise wording might be “correct for variations in the distribution of precipitation rates in the training data relative to comparable ground validation data.”
- Line 465: Replace “stronger” with “more strongly”.
- Line 473: Replace “small” with “low”.
- Line 475: Constant in time, space, or both?
- Line 485: Please define “GPM CO” in line 69 so the acronym is already defined.
- Lines 485-486: the latitude range of the GPM DPR is actually 65â°S to 65â°N when the instrument swath is accounted for.
- Line 489: This is the first time that the CNN is described as a “probabilistic regression” approach; this concept should be introduced earlier in the manuscript.
- Line 494: Delete the comma after “resolutions”.
- Line 587: “The Python Language Foundation” should be considered as starting with “P” since “The” is ignored when alphabetizing entries.
Citation: https://doi.org/10.5194/egusphere-2022-78-RC2 - AC3: 'Reply on RC2', Simon Pfreundschuh, 19 Aug 2022
Peer review completion
Journal article(s) based on this preprint
Simon Pfreundschuh et al.
Simon Pfreundschuh et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 440 | 168 | 14 | 622 | 12 | 4 |
- HTML: 440
- PDF: 168
- XML: 14
- Total: 622
- BibTeX: 12
- EndNote: 4
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(9103 KB) - Metadata XML