the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 14 Nov 2022
CLGAN: A GAN-based video prediction model for precipitation nowcasting
Abstract. The prediction of precipitation patterns at high spatio-temporal resolution up to two hours ahead, also known as precipitation nowcasting, is of great relevance in weather-dependant decision-making and early warning systems. In this study, we are aiming to provide an efficient and easy-to-understand model - CLGAN, to improve the nowcasting skills of heavy precipitation events with deep neural networks for video prediction. The model constitutes a Generative Adversarial Network (GAN) architecture whose generator is built upon an u-shaped encoder-decoder network (U-Net) equipped with recurrent LSTM cells to capture spatio-temporal features. A comprehensive comparison among CLGAN, and baseline models optical flow model DenseRotation as well as the advanced video prediction model PredRNN-v2 is performed. We show that CLGAN outperforms in terms of scores for dichotomous events and object-based diagnostics. The ablation study indicates that the GAN-based architecture helps to capture heavy precipitation events. The results encourage future work based on the proposed CLGAN architecture to improve the precipitation nowcasting and early-warning systems.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(2638 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2638 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
CEC1: 'Comment on egusphere-2022-859', Juan Antonio Añel, 12 Dec 2022
Dear authors,
Unfortunately, after checking your manuscript, it has come to our attention that it does not comply with our "Code and Data Policy".
https://www.geoscientific-model-development.net/policies/code_and_data_policy.htmlYour manuscript states that you have archived the code and data necessary to replicate your study in Zenodo. However, the text does not indicate how to reach such a repository; therefore, this is the same that does not publish it.
Your manuscript should have never been published in Discussions with such a shortcoming. However, now that this oversight by the Topical Editor has happened, we are offering you the possibility to solve it. In this way, you must publish a reply to this comment as soon as possible with the link and DOi of the suitable repository, according to our policy, where the code and data are stored.
Otherwise, we will have to end the Discussion stage and reject your manuscript for publication.
Best regards,
Juan A. Añel
Geosci. Model Dev. Executive Editor
Citation: https://doi.org/10.5194/egusphere-2022-859-CEC1 -
AC1: 'Reply on CEC1', Yan Ji, 12 Dec 2022
Dear Editor,
Thanks a lot for your comments. Sorry for the inconvenience.
Please find the link to our repository and data used in this paper here:
https://zenodo.org/record/7278016#.Y2ppyL7MIuQ
with the following doi: 10.5281/zenodo.7278016Indeed, the file we uploaded on 09 Nov. 2022 includes a hyperlink (with a hidden link to zotero and not shown in sentence instead) to point to our code repo and data. And we just noticed that the link disappears in our preprint version.
Again, sorry for this inconvenience. I hope everything works fine now.Best regards,
Yan Ji on behalf of all Co-AuthorsCitation: https://doi.org/10.5194/egusphere-2022-859-AC1
-
AC1: 'Reply on CEC1', Yan Ji, 12 Dec 2022
-
RC1: 'Comment on egusphere-2022-859', Anonymous Referee #1, 13 Dec 2022
Review of Ji et al., CLGAN: A GAN-based video prediction model for precipitation nowcasting
Ji et al. present a method combining a generative adversarial network (GAN) with a U-Net and recurrent LSTM cells as the generator, for a high spatiotemporal resolution prediction of precipitation up to two hours ahead. The CLGAN model introduced in this work is compared against a set of baseline models and competing deep-learning based models with a comprehensive set of evaluation criteria. The work is well presented and the evaluation appears comprehensive, and is good for the scope of GMD. I only have minor comments before recommending this manuscript for publication.
General comments:
1. The authors use an “ablation study” (as defined in the abstract), or more specifically a sensitivity analysis on the weight of the GAN component, to assess the importance of the newly added GAN component for generating forecasts that pass the visual test and are more skillful in capturing the statistical properties of observed precipitation. This is a very useful contribution, although I feel that too many different technical terms are used in different places of the manuscript for describing this process (L8-10; L53-57; L73-74; L331-L342). I recommend the authors start with more general terms (e.g., “weight of the GAN-component”) then use the terms reconstruction los and adversarial loss after they are introduced in L189 eq. (3). Using it too early in the introduction may confuse readers.
2. Through looking at the code archived on Zenodo there appears to be code from a Git repository. Besides providing the Zenodo DOI could you also provide the GitLab repository link as well? This way potential users can follow the developments of the code and look at the README and documentation easier.Specific comments:
1. L73-74: As in general comment 1, try to avoid terms like “adversarial loss” before introducing it. I also suggest being more specific about “sheds light on the interaction between the generator and the discriminator”, e.g., the role of the GAN-component in generating forecasts with closer statistical properties of the observed precipitation.
2. L161-162: The authors apply stronger weighting on higher precipitation rates to optimize towards heavy precipitation events. How much of an effect (if any) does this have on the precipitation of lighter precipitation events?
3. L372-374: It is mentioned that CLGAN can provide probabilistic forecasting by adding random noises. There is also mention of using ensemble forecasts to quantify forecast uncertainty in the introduction (L49-51). Is this part of future work or shown in this work? It was not very clear and I had to look if probabilistic forecasts were made in this work.
4. Figure 1(c). Is L^G supposed to be L^GAN here, to be consistent with text?
5. Figure 6. For ease to read please label the subfigures (a)-(e) with subtitles (Observation, Persistence, DenseRotation, …)Citation: https://doi.org/10.5194/egusphere-2022-859-RC1 - AC2: 'Reply on RC1', Yan Ji, 17 Jan 2023
-
RC2: 'Comment on egusphere-2022-859', Anonymous Referee #2, 16 Dec 2022
The paper presents an interesting new method for nowcasting of precipitation, as well as a new precipitation dataset which could be useful for future machine learning applications. The work is thorough and well thought out, and would be suitable for publication in GMD after a few modifications and clarification listed below:
Major Comments
- There are a couple of instances where the paper would benefit from an additional grammar check. This is especially noticeable in the abstract, the first few paragraphs of the introduction and the conclusion. I have highlighted some specific examples in my minor comments, but I would advise the authors to check the prepositions they use throughout the paper.
- I find it odd that in Figure 3, the models cannot outperform the persistence forecast. Surely the ConvLSTM should be able to at least match the persistence as in effect one of its inputs is the persistence forecast. Do you have any intuition as to why you cannot beat persistence?
- In Figure 3c) and 3d) there is a very large degradation in skill at the 20 minute lead time for the ConvLSTM. As far as I can tell this is not discussed in the work and I think it needs to be discussed as it is quite a stark difference.
- Lines 286-288: It would be nice to see some discussion here as to why CLGAN is superior in terms of dichotomous forecast scores but not for RMSE. What attributes does it have or do other models not have, which help here?
- Line 329: You say that CLGAN is doing better than ConvLSTM in the difference plot but it is different to see this in the eyeball norm. It would be useful to have some metrics even if it is just RMSE. Also please clarify what you mean by “fine cells”
- Line 331: The topic change here is very confusing as I thought you were still talking about the case study. Maybe add a sub-seciton title
- Lines 370-371 “It shows…”: This sentence is very unclear. Have you already tried adding additional predictors? If so please provide a reference to this work. If this is future work then the sentence needs to be rephrased because currently it reads like this is a conclusion of the paper
- Lines 371-372 “In addition…”: Please explain further what you mean by expanding to probabilistic forecasting by adding random noise and how you could do this with your model.
Minor Comments
- Lines 6-8: This sentence is very difficult to understand and needs to be rephrased
- Lines 73-74: The last highlight is very difficult to understand and needs to be rephrased
- Lines 100-102: This sentence needs to be rephrased
- Lines 120-121: This sentence needs to be rephrased
- Section 3.1.4: It would be helpful to put references to figure 1 in this section because it is very difficult to follow the CLGAN structure without references to the figure
- Line 242: You are missing the index i in the expression and it is very confusing to have the forecast expression with the observation expression in brackets. You should separate out the two expressions.
- Figure 4: Please clarify in the text and figure caption what the box and whiskers represent
- Figure 5: Add units to the legend and mention what the legend is in the caption
- Figure 6: It is interesting that ConvLSTM does better at longer lead times here given that it does worse at longer lead times in the metrics. Do you have any intuition on this? Is it just a quirk of the case study you chose?
- Line 344: This line needs to be rephrased
- Line 358: More potential than what?
- Section 5: It would be interesting to have a comment about how you think your model would perform at longer lead times (say 6hrs). Would you still see such good results?
Citation: https://doi.org/10.5194/egusphere-2022-859-RC2 - AC3: 'Reply on RC2', Yan Ji, 17 Jan 2023
-
CC1: 'Comment on egusphere-2022-859', Qiuming Kuang, 20 Dec 2022
This paper presents a method of CLGAN（Convolutional Long short-term memory Generative Adversarial Network） for precipitation nowcasting. Experiment proves that the method is effective in capturing heavy rainfall events, which is very important for disaster prevention and mitigation. Meanwhile, the authors shared a precipitation data set from 2015 to 2019. This work is clearly presented. A few commends listed below:
1.DGMR (Skilful precision nowcasting using deep generating models of radar) is a SOTA algorithm for precipitation prediction using GAN method. DGMR uses radar echo data, while CLGAN does not use radar echo data. If conditions permit, it is suggested that CLGAN and DGMR methods can be compared. Otherwise, please compare and explain the advantages and disadvantages of the two methods.
2.In this paper, the authors point out that this method can improve heavy precipitation prediction. However, it is necessary to consider the strong radar echo, dynamic, water vapor, thermal and other environmental conditions in order to make a accurate heavy precipitation forecast. The authors are suggested to express this point.
3.Figure 1 is somewhat miss-leading. In current version, the readers know how to get the t+1 th prediction using past m observations. However, the following n-1 frames are not provided. Certainly the results can be obtained iteratively. It is better to illustrate this explictly.
4.In Figure 1, the input channel is c. It is not clear what is the actual number of c. And how many kinds of inputs are embeded.
Citation: https://doi.org/10.5194/egusphere-2022-859-CC1 - AC4: 'Reply on CC1', Yan Ji, 17 Jan 2023
Interactive discussion
Status: closed
-
CEC1: 'Comment on egusphere-2022-859', Juan Antonio Añel, 12 Dec 2022
Dear authors,
Unfortunately, after checking your manuscript, it has come to our attention that it does not comply with our "Code and Data Policy".
https://www.geoscientific-model-development.net/policies/code_and_data_policy.htmlYour manuscript states that you have archived the code and data necessary to replicate your study in Zenodo. However, the text does not indicate how to reach such a repository; therefore, this is the same that does not publish it.
Your manuscript should have never been published in Discussions with such a shortcoming. However, now that this oversight by the Topical Editor has happened, we are offering you the possibility to solve it. In this way, you must publish a reply to this comment as soon as possible with the link and DOi of the suitable repository, according to our policy, where the code and data are stored.
Otherwise, we will have to end the Discussion stage and reject your manuscript for publication.
Best regards,
Juan A. Añel
Geosci. Model Dev. Executive Editor
Citation: https://doi.org/10.5194/egusphere-2022-859-CEC1 -
AC1: 'Reply on CEC1', Yan Ji, 12 Dec 2022
Dear Editor,
Thanks a lot for your comments. Sorry for the inconvenience.
Please find the link to our repository and data used in this paper here:
https://zenodo.org/record/7278016#.Y2ppyL7MIuQ
with the following doi: 10.5281/zenodo.7278016Indeed, the file we uploaded on 09 Nov. 2022 includes a hyperlink (with a hidden link to zotero and not shown in sentence instead) to point to our code repo and data. And we just noticed that the link disappears in our preprint version.
Again, sorry for this inconvenience. I hope everything works fine now.Best regards,
Yan Ji on behalf of all Co-AuthorsCitation: https://doi.org/10.5194/egusphere-2022-859-AC1
-
AC1: 'Reply on CEC1', Yan Ji, 12 Dec 2022
-
RC1: 'Comment on egusphere-2022-859', Anonymous Referee #1, 13 Dec 2022
Review of Ji et al., CLGAN: A GAN-based video prediction model for precipitation nowcasting
Ji et al. present a method combining a generative adversarial network (GAN) with a U-Net and recurrent LSTM cells as the generator, for a high spatiotemporal resolution prediction of precipitation up to two hours ahead. The CLGAN model introduced in this work is compared against a set of baseline models and competing deep-learning based models with a comprehensive set of evaluation criteria. The work is well presented and the evaluation appears comprehensive, and is good for the scope of GMD. I only have minor comments before recommending this manuscript for publication.
General comments:
1. The authors use an “ablation study” (as defined in the abstract), or more specifically a sensitivity analysis on the weight of the GAN component, to assess the importance of the newly added GAN component for generating forecasts that pass the visual test and are more skillful in capturing the statistical properties of observed precipitation. This is a very useful contribution, although I feel that too many different technical terms are used in different places of the manuscript for describing this process (L8-10; L53-57; L73-74; L331-L342). I recommend the authors start with more general terms (e.g., “weight of the GAN-component”) then use the terms reconstruction los and adversarial loss after they are introduced in L189 eq. (3). Using it too early in the introduction may confuse readers.
2. Through looking at the code archived on Zenodo there appears to be code from a Git repository. Besides providing the Zenodo DOI could you also provide the GitLab repository link as well? This way potential users can follow the developments of the code and look at the README and documentation easier.Specific comments:
1. L73-74: As in general comment 1, try to avoid terms like “adversarial loss” before introducing it. I also suggest being more specific about “sheds light on the interaction between the generator and the discriminator”, e.g., the role of the GAN-component in generating forecasts with closer statistical properties of the observed precipitation.
2. L161-162: The authors apply stronger weighting on higher precipitation rates to optimize towards heavy precipitation events. How much of an effect (if any) does this have on the precipitation of lighter precipitation events?
3. L372-374: It is mentioned that CLGAN can provide probabilistic forecasting by adding random noises. There is also mention of using ensemble forecasts to quantify forecast uncertainty in the introduction (L49-51). Is this part of future work or shown in this work? It was not very clear and I had to look if probabilistic forecasts were made in this work.
4. Figure 1(c). Is L^G supposed to be L^GAN here, to be consistent with text?
5. Figure 6. For ease to read please label the subfigures (a)-(e) with subtitles (Observation, Persistence, DenseRotation, …)Citation: https://doi.org/10.5194/egusphere-2022-859-RC1 - AC2: 'Reply on RC1', Yan Ji, 17 Jan 2023
-
RC2: 'Comment on egusphere-2022-859', Anonymous Referee #2, 16 Dec 2022
The paper presents an interesting new method for nowcasting of precipitation, as well as a new precipitation dataset which could be useful for future machine learning applications. The work is thorough and well thought out, and would be suitable for publication in GMD after a few modifications and clarification listed below:
Major Comments
- There are a couple of instances where the paper would benefit from an additional grammar check. This is especially noticeable in the abstract, the first few paragraphs of the introduction and the conclusion. I have highlighted some specific examples in my minor comments, but I would advise the authors to check the prepositions they use throughout the paper.
- I find it odd that in Figure 3, the models cannot outperform the persistence forecast. Surely the ConvLSTM should be able to at least match the persistence as in effect one of its inputs is the persistence forecast. Do you have any intuition as to why you cannot beat persistence?
- In Figure 3c) and 3d) there is a very large degradation in skill at the 20 minute lead time for the ConvLSTM. As far as I can tell this is not discussed in the work and I think it needs to be discussed as it is quite a stark difference.
- Lines 286-288: It would be nice to see some discussion here as to why CLGAN is superior in terms of dichotomous forecast scores but not for RMSE. What attributes does it have or do other models not have, which help here?
- Line 329: You say that CLGAN is doing better than ConvLSTM in the difference plot but it is different to see this in the eyeball norm. It would be useful to have some metrics even if it is just RMSE. Also please clarify what you mean by “fine cells”
- Line 331: The topic change here is very confusing as I thought you were still talking about the case study. Maybe add a sub-seciton title
- Lines 370-371 “It shows…”: This sentence is very unclear. Have you already tried adding additional predictors? If so please provide a reference to this work. If this is future work then the sentence needs to be rephrased because currently it reads like this is a conclusion of the paper
- Lines 371-372 “In addition…”: Please explain further what you mean by expanding to probabilistic forecasting by adding random noise and how you could do this with your model.
Minor Comments
- Lines 6-8: This sentence is very difficult to understand and needs to be rephrased
- Lines 73-74: The last highlight is very difficult to understand and needs to be rephrased
- Lines 100-102: This sentence needs to be rephrased
- Lines 120-121: This sentence needs to be rephrased
- Section 3.1.4: It would be helpful to put references to figure 1 in this section because it is very difficult to follow the CLGAN structure without references to the figure
- Line 242: You are missing the index i in the expression and it is very confusing to have the forecast expression with the observation expression in brackets. You should separate out the two expressions.
- Figure 4: Please clarify in the text and figure caption what the box and whiskers represent
- Figure 5: Add units to the legend and mention what the legend is in the caption
- Figure 6: It is interesting that ConvLSTM does better at longer lead times here given that it does worse at longer lead times in the metrics. Do you have any intuition on this? Is it just a quirk of the case study you chose?
- Line 344: This line needs to be rephrased
- Line 358: More potential than what?
- Section 5: It would be interesting to have a comment about how you think your model would perform at longer lead times (say 6hrs). Would you still see such good results?
Citation: https://doi.org/10.5194/egusphere-2022-859-RC2 - AC3: 'Reply on RC2', Yan Ji, 17 Jan 2023
-
CC1: 'Comment on egusphere-2022-859', Qiuming Kuang, 20 Dec 2022
This paper presents a method of CLGAN（Convolutional Long short-term memory Generative Adversarial Network） for precipitation nowcasting. Experiment proves that the method is effective in capturing heavy rainfall events, which is very important for disaster prevention and mitigation. Meanwhile, the authors shared a precipitation data set from 2015 to 2019. This work is clearly presented. A few commends listed below:
1.DGMR (Skilful precision nowcasting using deep generating models of radar) is a SOTA algorithm for precipitation prediction using GAN method. DGMR uses radar echo data, while CLGAN does not use radar echo data. If conditions permit, it is suggested that CLGAN and DGMR methods can be compared. Otherwise, please compare and explain the advantages and disadvantages of the two methods.
2.In this paper, the authors point out that this method can improve heavy precipitation prediction. However, it is necessary to consider the strong radar echo, dynamic, water vapor, thermal and other environmental conditions in order to make a accurate heavy precipitation forecast. The authors are suggested to express this point.
3.Figure 1 is somewhat miss-leading. In current version, the readers know how to get the t+1 th prediction using past m observations. However, the following n-1 frames are not provided. Certainly the results can be obtained iteratively. It is better to illustrate this explictly.
4.In Figure 1, the input channel is c. It is not clear what is the actual number of c. And how many kinds of inputs are embeded.
Citation: https://doi.org/10.5194/egusphere-2022-859-CC1 - AC4: 'Reply on CC1', Yan Ji, 17 Jan 2023
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 833 | 288 | 30 | 1,151 | 18 | 17 |
- HTML: 833
- PDF: 288
- XML: 30
- Total: 1,151
- BibTeX: 18
- EndNote: 17
Viewed (geographical distribution)
| Country | # | Views | % |
|---|---|---|---|
| China | 1 | 466 | 41 |
| United States of America | 2 | 216 | 19 |
| Germany | 3 | 125 | 11 |
| United Kingdom | 4 | 36 | 3 |
| France | 5 | 35 | 3 |
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
- 466
Cited
Yan Ji
Michael Langguth
Amirpasha Mozaffari
Xiefei Zhi
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2638 KB) - Metadata XML