the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 26 Jan 2026
Scalable radar-driven approach with compact gradient-boosting models for gap filling in high-resolution precipitation measurements
Abstract. High-frequency precipitation records are essential for hydrological modeling, weather forecasting, and ecosystem research. Unfortunately, they usually exhibit data gaps originating from sensor malfunctions, significantly limiting their usability. We present a framework to reconstruct missing data in precipitation measurements sampled at 10 min frequency using radar-based, gauge independent, precipitation estimates as the only predictor. We fit gradient-boosting models to the statistical relationships between radar-based precipitation fields and collocated rain gauges. The obtained models allow for the filling of data gaps of arbitrary length and additionally provide confidence interval approximations. We evaluate the method using the rain gauge network of the German Weather Service (DWD), which roughly covers the entirety of Germany. The results show robust performance across diverse climatic and topographic conditions at a high level, with the coefficient of determination averaging at around 0.7. The framework is computationally very cheap, relying on a single CPU core only. This makes scaling easy and integration into operational gap filling of extensive sensor networks feasible.
- Preprint
(4634 KB) - Metadata XML
- BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on egusphere-2025-6349', Anonymous Referee #1, 08 May 2026
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2026/egusphere-2025-6349/egusphere-2025-6349-RC1-supplement.pdfCitation: https://doi.org/
10.5194/egusphere-2025-6349-RC1 -
AC1: 'Reply on RC1', Peter Lünenschloß, 16 Jul 2026
Major Comment 1:
The authors took great care to put their methods in broader context and provided a thorough overview of existing solutions for measurement gap filling problems. In the light of this, I find it surprising that the results are not explicitly compared to any other method: authors just provide the values of several standard statistical tests and claim that they are good. I fully understand that implementing alternative approaches for the particular example that the authors used here might be laborious, but perhaps some key quantitative results from some of the numerous cited works could be quoted in the main text? The claim that the values of the statistical parameters used to evaluate the new method’s performance are “good”, must, after all, be based on a comparison of some kind? Even if some of the statistical tests are standard, basic comparisons to other works would save a non-expert reader from having to dig into all the references.
Response:
We thank the reviewer for this helpful suggestion. We agree that the reported performance values should be placed in a clearer quantitative context. We will expand Sect. 5.1 by adding a contextual comparison table with representative R2 values from previously cited sub-hourly precipitation-imputation studies such as Chivers et al. (2020) and Vidal-Paz et al. (2023), while explicitly stating that this comparison is not a formal head-to-head benchmark because datasets, predictors, gap structures, temporal resolutions, and evaluation protocols differ between studies.
We also added an interpretive paragraph explaining that our results are comparable to the upper range reported by Chivers et al. (2020), although their best-performing configurations used neighbouring rain gauges and additional local meteorological predictors. For Vidal-Paz et al. (2023), who also considered 10 min precipitation data, the reported mean R2 values are lower, although individual maximum values approach the lower range of our results. Against this background, we now interpret our mean R2 = 0.66 for 10 min gaps and values above 0.55 for longer or arbitrary gaps as strong performance for a radar-only setup evaluated across a large and heterogeneous national network.
Major Comment 2
Figures 4 and 7 give an impression that the rainfall from some short and intense rain periods (which authors attribute to “highly convective or erratic rainfall events”) can be significantly underrepresented. This may be a problem for some of the applications, but has not been explicitly quantified. Is the performance seen in Figure 4 and Figure 7 typical? Is there an overall low bias in rainfall predictions? Is there a bias in case of short periods of intense rain? These questions seem particularly relevant, since authors suggest to use the new method for runoff simulations and flash-flood prediction.
Response
We thank the reviewer for pointing out that short and intense rainfall events may be underrepresented and that this is particularly relevant for hydrological downstream applications. We will revise the manuscript to clarify the scope of the proposed framework more explicitly. The method will be described as a gap-filling and imputation approach rather than as a convective nowcasting or stand-alone flash-flood prediction system. Accordingly, we will soften the application-related wording in Sect. 5.4 and Sect. 5.5 and state that the framework may support downstream runoff modelling, early-warning, and flood-forecasting workflows by providing more complete high-frequency precipitation input data, rather than by directly predicting such events.
Furthermore, we will evaluate and quantify the imputation accuracy for more intense precipitation events in a more transparent manner by adding a supplementary table with suitable performance metrics. Since we do not have access to a convective-event classification for our data set, we will instead evaluate model performance for increasingly intense precipitation using rainfall-intensity thresholds. More specifically, we will report selected classification metrics (HSS, CSI, and SEDI), as well as regression metrics (CRMSE and CMAPE), for subsets of the data where the observed rainfall intensity exceeds 0.1 mm, 1.0 mm, and 5.0 mm, respectively.
As of now, we only have the additional regression scores readily evaluated at the different threshing levels. The results are compiled in supplement tables 1 and 2.
Although rainfall intensity is not equivalent to convective precipitation, this analysis provides a first indication and quantification of how model performance changes under increasingly challenging precipitation conditions and confirms the expected reduction in reconstruction performance. Across both investigated gap lengths, Pearson correlation decreases progressively, while RMSE and MAPE increase as higher rainfall thresholds are considered. These results quantitatively support our previous discussion that short-lived and highly intense rainfall events are inherently more difficult to reconstruct than moderate precipitation events.
Major Comment 3
The authors claim that the method has low computational costs and good scalability. Unless I have missed something, the only concrete data regarding overall computational cost was the statement that “end-to-end processing for one station” required around 15 min on a single CPU core “with 30 GB of assigned RAM”. This information is sufficient to convince me that computational costs are indeed quite low, and thus the method is in principle suitable for processing large data sets and/or real time applications. However, more information would be needed if claims about scalability are made. Firstly, was the model really constrained to a single CPU core, or was it simply not (explicitly) parallelized? Modern programming languages often have significant implicit parallelization capabilities. Secondly, while 30 GB of memory was assigned, how much was actually used? This is important, because if the model indeed runs on a single CPU core and requires 30 GB of memory, it is a rather memory-intensive application. Both modern workstations and HPC systems typically have way less than 30 GB of memory per CPU core, and would, in this case, struggle to process many stations in parallel. Finally, since not all 1 CPU cores are equally powerful, it would be good to specify the model of CPU used here
Response:
We thank the reviewer for pointing out that the computational requirements of the proposed framework were described too briefly. Since computational scalability and practical feasibility are intended to be among the main strengths of the approach, we agree that this aspect deserves a more transparent and quantitative discussion.
First, we will clarify our CPU specification. Since the term CPU core is used ambiguously in both the literature and casual communication, we will explicitly state that all runtimes reported in the manuscript refer to execution on a single physical CPU core. Furthermore, this restriction applies to the complete execution chain, including all secondary library calls made during model training and prediction.
At the same time, our Python implementation makes use of libraries that internally support parallel execution (primarily NumPy, Cython, and XGBoost). We will therefore complement the manuscript with runtime measurements obtained using 2, 4, and 8 CPU cores for the modelling of a representative station in order to illustrate the achievable reduction in computation time.
Since the modelling workflow was executed automatically on the UFZ HPC cluster, different stations may have been assigned different processors from the available hardware pool. The available processors (ranked according to their single-thread performance) were:
-
Intel Xeon 8280 (4.0 GHz)
-
Intel Xeon 6710E (3.2 GHz)
-
Intel Xeon E7-8867 (3.3 GHz)
We will therefore also state that the reported single-core runtimes remained below 15 minutes even when execution was assigned to the slowest processor available (Intel Xeon E7-8867), thereby providing a conservative estimate of the computational cost.
With regard to memory consumption, the actual peak memory usage during model training was approximately 15 GB. While this is still substantial, it should be noted that the current implementation was not specifically optimized for memory efficiency. In particular, intermediate data structures are not released as aggressively as possible, and the lagged feature construction currently relies on materializing the complete feature matrix rather than generating features on demand. Consequently, the reported memory consumption should be regarded as a conservative estimate of the implementation rather than an inherent requirement of the proposed methodology.
Finally, we agree that memory availability may become a more relevant limiting factor than CPU time when modelling multiple stations simultaneously. We will therefore extend the computational-cost analysis by including runtimes for parallel execution using multiple CPU cores. The resulting reduction in processing time (already below five minutes for eight CPU cores) provides an additional option for balancing runtime against the available computational resources.
Since each station is modelled independently, the computational workload is embarrassingly parallel across stations. Consequently, increasing the number of available compute nodes reduces wall-clock time almost linearly until the number of stations becomes the limiting factor.
-
-
AC1: 'Reply on RC1', Peter Lünenschloß, 16 Jul 2026
-
RC2: 'Comment on egusphere-2025-6349', Anonymous Referee #2, 11 May 2026
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2026/egusphere-2025-6349/egusphere-2025-6349-RC2-supplement.pdf
-
AC2: 'Reply on RC2', Peter Lünenschloß, 16 Jul 2026
Major Comment 1
Section 2.2: Please provide additional detail regarding the 17 dual-polarization DWD weather radars, including radar band (S-, C-, or X-band), range and azimuthal gate spacing, effective beamwidth, maximum range, and the extent to which the radar beams clear the local horizon (e.g., lowest usable elevation angle, maximum elevation angle, and terrain blockage considerations). These characteristics can substantially influence QPE quality and uncertainty. Radar-derived QPE is often sensitive to factors such as beam geometry, attenuation, beam blockage, storm mode, bright band contamination, and sampling resolution, all of which may lead to over- or underestimation of rainfall. Given that the radar-based QPE retrievals are a major component of this manuscript, the current description does not provide enough detail about the radar systems themselves or the quality-control and validation methodologies used to ensure reliable precipitation estimates across different storm types and geographic regions. These are only slightly mentioned later in the manuscript and often glossed over (e.g., lines 280-290, 409-417). Elaborate how you all combat these issues with these specific radars.
Response:
We thank the reviewer for pointing out that the radar input requires a more explicit description. We expanded Sect. 2.2 and added a new table summarising the main characteristics of the DWD radar input used for QPECommon, including radar band, native polar sampling, effective beamwidth, maximum range, scan strategy, elevation range, and terrain-related limitations.
The DWD precipitation scan is a terrain-following PPI scan with azimuth-dependent elevation angles rather than a single fixed low-level elevation. The configured low standard elevation is 0.8°, the precipitation-scan range is 150 km, and the operational volume scan covers fixed elevations between 0.5° and 25°. We will also expand the discussion of radar-QPE uncertainty and explain that several known error sources are already mitigated in the DWD processing chain. This chain includes measures such as quality control of non-meteorological echoes, the incorporation of dual-polarisation-based hydrometeor information, melting-layer detection with Hymec, melting-layer-aware Z-R conversion, and the weighted compositing of overlapping radar domains. (The steps are compiled in supplement table 3.)
At the same time, we make it clearer that we do not apply additional physical corrections for attenuation, beam blockage, vertical-profile effects, or storm-type-dependent radar errors in this study, as we do not aim to calibrate and deliver a final spatial precipitation product. Instead, we aim to evaluate the usability of QPECommon as a gauge-independent predictor, which excludes several critical gauge-supported calibrations and corrections.
Residual radar errors are therefore treated as predictor uncertainty and are addressed through station-specific aggregates of spatiotemporal vicinities during model training, as well as through our correlation-based reference-pixel selection and spatial QPE predictor aggregation during feature preprocessing. Our final prediction evaluation against the point-wise available ground truth, as represented by the DWD rain gauge network, also serves as a measure of how successful this indirect treatment of the remaining radar errors has been.
We are aware that the performance of the proposed framework is ultimately bounded by the information content and uncertainty structure of its sole predictor, QPECommon. Although the DWD radar processing chain mitigates several known error sources, QPECommon remains an ungauged radar-only product, and residual errors and biases can only be mitigated by our approach insofar as they occur with sufficient regularity and frequency in our training data. This especially means that capturing highly convective or erratic rainfall events remains a challenge, as these are inherently less predictable and are often underrepresented in short training periods. Furthermore, the available data record for training and testing was limited to three years at the time the study was set up.
Major Comment 2-1
Lines 367-368: Why were observations from 2024 specifically selected for the final model evaluation? Was this year chosen randomly (which it should be), or was there a particular rationale for its selection?
Response
We realize that we might have brushed over the conditions and the context underlying the selection and design of our modelling and, especially, model evaluation approach, and are happy to make this process more transparent.
At the time of setting up the study, three years of complete data records were available (2022–2024). The rationale guiding the separation of the available data into training and test data was as follows:
Starting with the question of why we did not randomly sample one third of the data points from the complete data record to be set aside as test data:
Although our approach resorts to an ML framework that is based on tabular data input, through feature lagging we actually include 70-minute chunks of the target precipitation time series in every sample (30 minutes of measurements prior to and 30 minutes of data following the target precipitation value, which, of course, is masked). Thus, we implicitly construct a time-series-based framework. Consequently, our evaluation follows a blocked temporal hold-out rather than a random hold-out, as a random split would introduce temporal information leakage.
More specifically, in order to avoid leakage, we cannot include one sample in the training data and another sample within a 30-minute range in the test data. Due to the domino-like propagation of this restriction, a random sampling strategy would leave many observations unusable because neighbouring samples could no longer be assigned independently. This would substantially reduce the effective amount of data available for both training and testing and impair the representativeness of the resulting subsets. Even though there would still be short chunks of two or three subsequent periods assigned either to the training or the hold-out data, a large proportion of samples would have to be separated by at least one hour from the next sampled observation.
This leaves us with sampling continuous chunks of data with non-overlapping combined sample extensions and assigning those continuous chunks either to the training or the test data. Furthermore, to ensure that the test data are representative of model performance throughout an entire year and across all seasons, we chose blocks that each extend over one complete year. While we are aware that a single year cannot represent the full variability of each season, the available data volume did not permit setting aside more than one year's worth of observations for testing.
The selection of the final year of the available data record (2024) for testing was not made randomly. At the same time, it was also not pre-evaluated or compared against alternative train-test year permutations. The rationale for selecting the most recent year was that it best reflects the intended operational application of the imputation framework. In practice, the model will almost always be trained on historical observations and subsequently applied to future data. Evaluating the framework on the chronologically last year of the available data record therefore provides the most realistic estimate of the performance that can be expected during operational use.
Major Comment 2-2
Additionally, do the results shown in Table 2 remain similar if a different evaluation year is used?
Response
We agree that this information would be valuable, as it would provide further insight into the dependency of the evaluation metrics on the temporal separation between the training and test data sets.
Unfortunately, with only three consecutive years of data available, changing the evaluation year is not simply a matter of shifting the test period. Instead, it would require replacing one of the training years with the alternative test year and retraining the model accordingly. Such an analysis would therefore constitute an additional set of experiments rather than a straightforward variation of the existing evaluation.
Having said that, we did not evaluate how the performance metrics change when a different year is used for testing. We agree that this is an interesting question and will acknowledge this limitation more explicitly in the revised manuscript.
Major Comment 3
I am also somewhat concerned that the statistics presented in Table 2 may obscure model performance for the most societally impactful events, such as high-frequency and/or high-accumulation precipitation cases, because these events are averaged together with non-precipitation or weak-precipitation periods. Since extreme convective rainfall events are often of greatest socioeconomic importance, it would strengthen the manuscript to include a more targeted evaluation of AI/machine learning performance during these high-impact events. For example, it would be helpful to examine how the reported statistics change when only strongly convective or heavy-rainfall cases are considered. This concern also relates to Lines 427–430, where the authors note reduced robustness for such events. However, the current figures and tables do not clearly quantify or highlight this degradation in performance, making the issue easy to overlook.
Response
We thank the reviewer for this valuable suggestion and agree that the aggregated statistics presented in Table 2 do not sufficiently illustrate model performance during the most challenging precipitation events.
To address the reviewer's concern, we would like to complement the evaluation with an additional event-focused analysis that quantifies model performance for increasingly intense rainfall conditions. Since no objective classification of convective events is available for the study period, we instead evaluate subsets of observations exceeding several rainfall-intensity thresholds. The corresponding results will be added as supplementary tables (1 and 2) and than referenced in the revised Results and Discussion sections.
The additional evaluation confirms the expected reduction in reconstruction performance with increasing rainfall intensity. Across both investigated gap lengths, Pearson correlation decreases progressively, while RMSE and MAPE increase as higher rainfall thresholds are considered. These results quantitatively support our previous discussion that short-lived and highly intense rainfall events are inherently more difficult to reconstruct than moderate precipitation events.
In response to this comment, we have also revised the discussion of potential applications. The manuscript now more clearly describes the proposed framework as a radar-guided precipitation gap-filling and imputation approach rather than as a dedicated convective nowcasting or flash-flood prediction system. While more complete high-frequency precipitation records may support downstream runoff modelling, flood forecasting, or early-warning workflows, the proposed framework is not intended to directly predict or fully resolve highly convective precipitation events. The threshold-based evaluation provides a quantitative assessment of this limitation.
Minor Comment 1
A few times the authors describe the 10-minute precipitation dataset (or even data with a frequency of < 30 min; line 54) as “high-frequency” or “high-resolution” (e.g., lines 2, 13, 17, etc.). Some may not consider 10-min data to be truly high-frequency given that in-situ instruments and radar observations can provide data at 1-min, 10-Hz, or even finer temporal resolution. Consider rewording this terminology or clarifying the context in which the dataset is considered high-frequency and/or high-resolution. For example, the authors could emphasize that the description is relative to the temporal and spatial scales typically available in climatological or gridded precipitation datasets.
Response
We are thankfull for the hint and will specify the terminology as suggested.
Minor Comment 2
Line 3: Are data gaps truly only from “sensor malfunctions”? The authors do go into more detail in lines 27-31 about what causes data gaps, and not all are due to sensor malfunctions (i.e., outages, quality control, human error, etc.).
Response
We will widen the here noted scope of referenced causes accordingly.
Minor Comment 3
Line 57: The DWD acronym has not been introduced yet until line 60. Replace with “the German Weather Service (DWD) national gauge network” in line 57 and remove from line 60.
Response
Thanks for the hint. We didnt notice the abbreviation wasnt properly introduced with usage.
Minor Comment 4
Line 79: Is RADOLAN an acronym?
Response
Yes. Its pronounced as an actual word, abbreviating german for: ‘radar online calibration'.
-
AC2: 'Reply on RC2', Peter Lünenschloß, 16 Jul 2026
Data sets
Scalable radar-driven approach with compact gradient-boosting models for gap filling in high-resolution precipitation measurements Peter Lünenschloß et al. https://doi.org/10.5281/zenodo.17937464
Model code and software
Scalable radar-driven approach with compact gradient-boosting models for gap filling in high-resolution precipitation measurements Peter Lünenschloß et al. https://doi.org/10.5281/zenodo.17940311
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 931 | 974 | 97 | 2,002 | 110 | 221 |
- HTML: 931
- PDF: 974
- XML: 97
- Total: 2,002
- BibTeX: 110
- EndNote: 221
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
