the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 07 Apr 2025
How to deal w___ missing input data
Abstract. Deep learning hydrologic models have made their way from research to applications. More and more national hydrometeorological agencies, hydro power operators, and engineering consulting companies are building Long Short-Term Memory (LSTM) models for operational use cases. All of these efforts come across similar sets of challenges—challenges that are different from those in controlled scientific studies. In this paper, we tackle one of these issues: how to deal with missing input data? Operational systems depend on the real-time availability of various data products—most notably, meteorological forcings. The more external dependencies a model has, however, the more likely it is to experience an outage in one of them. We introduce and compare three different solutions that can generate predictions even when some of the meteorological input data do not arrive in time, or not arrive at all.
Competing interests: At least one of the (co-)authors is a member of the editorial board of Hydrology and Earth System Sciences. The peer-review process was guided by an independent editor, and the authors also have no other competing interests to declare.
Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors. Views expressed in the text are those of the authors and do not necessarily reflect the views of the publisher.-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(1314 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(1314 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
CC1: 'Comment on egusphere-2025-1224', Xin Yu, 07 Apr 2025
-
AC1: 'Reply on CC1', Martin Gauch, 24 Apr 2025
The title is a small joke that highlights how it's possible to infer a concept despite missing information. You probably figured out that the incomplete word is "with". Similarly, a neural network can make predictions when some of its inputs are missing.
Citation: https://doi.org/10.5194/egusphere-2025-1224-AC1
-
AC1: 'Reply on CC1', Martin Gauch, 24 Apr 2025
-
RC1: 'Comment on egusphere-2025-1224', Juliane Mai, 03 Jul 2025
Dear Dr. Gauch and others,
It was a pleasure to review your manuscript on “How to deal w___ missing input data”, submitted to the Hydrology and Earth System Science journal. I found your study to be well-structured, informative, and a very pleasant read. The experiments are clearly motivated, performed, and analyzed. I think this is a valuable contribution to the field.
I have provided a list of minor comments for your consideration in the attached PDF. Most of these do not require urgent revisions. Most comments are regarding clarity especially for readers that may not have much experience regarding methods like embedding and attention. The authors have already provided additional material which is much appreciated.
I have no doubts that the authors will be able to respond to all my comments without any problem. Hence, I am recommending minor revisions. I appreciate the effort you have put into this work and would be happy to have another look at the revised version.
Best regards,
Julie Mai- AC2: 'Reply on RC1', Martin Gauch, 05 Aug 2025
-
RC2: 'Comment on egusphere-2025-1224', Anonymous Referee #2, 10 Jul 2025
General comments
The paper “How to deal w___ missing input data” provides a thorough implementation and analysis of three strategies for how to deal with missing input data for operational AI-based models that depend on the real-time availability of meteorological forcings. The strategies are: input replacing, masked mean, and attention. Through three sets of experiments that train models with different permutations of missing forcings, they show that the masked mean strategy tends to perform the best, if only marginally. They show that attention appears to be unnecessary, as it is complicated and tends to collapse to the simpler weighted mean approach. This is a very nice result.
While the paper is well structured, the text requires some refinement. The description of the key experiments lack some details and make it difficult for the reader to follow. For instance, experiment 2 can only be understood after reading the figure caption, not from the text body. Furthermore, the reviewer has some concerns regarding the reproducibility, since the training code is not made available. Combined with some technical errors, a major revision is recommended.
Specific comments
Major concerns
- Code is missing. While the authors provide detailed code for the replication of the figures as well as the trained models, the code for the model training and inference is not available. In my opinion, and in line with GMD’s practices, the code should be made available before this manuscript can be published.
- The description of the LSTM model is missing in this manuscript. While the authors provide a thorough explanation of the model architectures for the input-data-processing methods, a description of the architecture for the core LSTM model is missing. The description of the target output of the LSTM model (i.e., what it is actually predicting) is also missing. i.e., Section 2.1 describes the forcing variables, but it is unclear what the target is.
- Section 2.1: Is there a mistake in the testing period? The testing period should not overlap with the training dataset; otherwise, this is a major error, as the models could be overfitting.
- Starting from section 3.2, following and understanding the experiment setup is difficult and needs the be clarified. For instance, in section 3.2, you mention that the results show the cases of missing forcings at inference. For a given line/plot, it is not clear what parts of each model are trained (just the LSTM, or also the data-preprocessing components?). It is not also clear what the difference is between the dashed lines and the masked mean/attention/input replacement lines are. I understood the dashed line as a LSTM model trained with 1 (or 2) forcings, and the solid lines are the same LSTM, but with an additional trained encoder (with the appropriate data-processing techniques). However, I could be misunderstanding this. For the dashed (reference) lines, how are the forcings passed to the LSTM (concat, mean, attention?).
Minor concerns and questions
- Figure 6: When you describe the attention mechanism for high probability p, it is an interesting result that the weights fluctuate around 1/3. Is this also true for the lower missing data probabilities, where the attention method performs worse? If not, what would your analysis be around this?
- From reading the manuscript, it is unclear to the reader why the metric becomes CDF for experiments 2 and 3. Please highlight the connection between these two metrics in the text, and specify that the ideal result is a delta function at NSE=1, and thus, curves that are closer to the right are better.
Specific, minor comments
- The abstract is missing key results. It would be beneficial to briefly mention the different solutions and hint at which provide the most robust results (seemingly, masked mean)
- Minor comment: for readability, it would be a smoother flow if the literature review section on other fields and models with missing input data occurs earlier in the introduction, before you present the three strategies to accomplish this goal.
Citation: https://doi.org/10.5194/egusphere-2025-1224-RC2 - AC3: 'Reply on RC2', Martin Gauch, 05 Aug 2025
Interactive discussion
Status: closed
-
CC1: 'Comment on egusphere-2025-1224', Xin Yu, 07 Apr 2025
Just curious. What is the intention of using 'w___' in the title?
Citation: https://doi.org/10.5194/egusphere-2025-1224-CC1 -
AC1: 'Reply on CC1', Martin Gauch, 24 Apr 2025
The title is a small joke that highlights how it's possible to infer a concept despite missing information. You probably figured out that the incomplete word is "with". Similarly, a neural network can make predictions when some of its inputs are missing.
Citation: https://doi.org/10.5194/egusphere-2025-1224-AC1
-
AC1: 'Reply on CC1', Martin Gauch, 24 Apr 2025
-
RC1: 'Comment on egusphere-2025-1224', Juliane Mai, 03 Jul 2025
Dear Dr. Gauch and others,
It was a pleasure to review your manuscript on “How to deal w___ missing input data”, submitted to the Hydrology and Earth System Science journal. I found your study to be well-structured, informative, and a very pleasant read. The experiments are clearly motivated, performed, and analyzed. I think this is a valuable contribution to the field.
I have provided a list of minor comments for your consideration in the attached PDF. Most of these do not require urgent revisions. Most comments are regarding clarity especially for readers that may not have much experience regarding methods like embedding and attention. The authors have already provided additional material which is much appreciated.
I have no doubts that the authors will be able to respond to all my comments without any problem. Hence, I am recommending minor revisions. I appreciate the effort you have put into this work and would be happy to have another look at the revised version.
Best regards,
Julie Mai- AC2: 'Reply on RC1', Martin Gauch, 05 Aug 2025
-
RC2: 'Comment on egusphere-2025-1224', Anonymous Referee #2, 10 Jul 2025
General comments
The paper “How to deal w___ missing input data” provides a thorough implementation and analysis of three strategies for how to deal with missing input data for operational AI-based models that depend on the real-time availability of meteorological forcings. The strategies are: input replacing, masked mean, and attention. Through three sets of experiments that train models with different permutations of missing forcings, they show that the masked mean strategy tends to perform the best, if only marginally. They show that attention appears to be unnecessary, as it is complicated and tends to collapse to the simpler weighted mean approach. This is a very nice result.
While the paper is well structured, the text requires some refinement. The description of the key experiments lack some details and make it difficult for the reader to follow. For instance, experiment 2 can only be understood after reading the figure caption, not from the text body. Furthermore, the reviewer has some concerns regarding the reproducibility, since the training code is not made available. Combined with some technical errors, a major revision is recommended.
Specific comments
Major concerns
- Code is missing. While the authors provide detailed code for the replication of the figures as well as the trained models, the code for the model training and inference is not available. In my opinion, and in line with GMD’s practices, the code should be made available before this manuscript can be published.
- The description of the LSTM model is missing in this manuscript. While the authors provide a thorough explanation of the model architectures for the input-data-processing methods, a description of the architecture for the core LSTM model is missing. The description of the target output of the LSTM model (i.e., what it is actually predicting) is also missing. i.e., Section 2.1 describes the forcing variables, but it is unclear what the target is.
- Section 2.1: Is there a mistake in the testing period? The testing period should not overlap with the training dataset; otherwise, this is a major error, as the models could be overfitting.
- Starting from section 3.2, following and understanding the experiment setup is difficult and needs the be clarified. For instance, in section 3.2, you mention that the results show the cases of missing forcings at inference. For a given line/plot, it is not clear what parts of each model are trained (just the LSTM, or also the data-preprocessing components?). It is not also clear what the difference is between the dashed lines and the masked mean/attention/input replacement lines are. I understood the dashed line as a LSTM model trained with 1 (or 2) forcings, and the solid lines are the same LSTM, but with an additional trained encoder (with the appropriate data-processing techniques). However, I could be misunderstanding this. For the dashed (reference) lines, how are the forcings passed to the LSTM (concat, mean, attention?).
Minor concerns and questions
- Figure 6: When you describe the attention mechanism for high probability p, it is an interesting result that the weights fluctuate around 1/3. Is this also true for the lower missing data probabilities, where the attention method performs worse? If not, what would your analysis be around this?
- From reading the manuscript, it is unclear to the reader why the metric becomes CDF for experiments 2 and 3. Please highlight the connection between these two metrics in the text, and specify that the ideal result is a delta function at NSE=1, and thus, curves that are closer to the right are better.
Specific, minor comments
- The abstract is missing key results. It would be beneficial to briefly mention the different solutions and hint at which provide the most robust results (seemingly, masked mean)
- Minor comment: for readability, it would be a smoother flow if the literature review section on other fields and models with missing input data occurs earlier in the introduction, before you present the three strategies to accomplish this goal.
Citation: https://doi.org/10.5194/egusphere-2025-1224-RC2 - AC3: 'Reply on RC2', Martin Gauch, 05 Aug 2025
Peer review completion
Journal article(s) based on this preprint
Data sets
Models, configs, and predictions Martin Gauch https://doi.org/10.5281/zenodo.15008460
Model code and software
GitHub repository with analysis code Martin Gauch https://github.com/gauchm/missing-inputs
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 1,082 | 199 | 25 | 1,306 | 27 | 40 |
- HTML: 1,082
- PDF: 199
- XML: 25
- Total: 1,306
- BibTeX: 27
- EndNote: 40
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Frederik Kratzert
Daniel Klotz
Grey Nearing
Deborah Cohen
Oren Gilon
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(1314 KB) - Metadata XML
Just curious. What is the intention of using 'w___' in the title?