the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 30 Jul 2026
Evaluating the performance of a numerical weather prediction model for precipitation and temperature in Luxembourg and the Greater Region: insights from WRF and WRFDA 3D-VAR
Abstract. This study evaluates the Weather Research and Forecasting (WRF) model, with and without WRFDA 3D-VAR data assimilation (DA), for precipitation and temperature forecasts in Luxembourg and the Greater Region during the July 2021 flood event. Conventional observations (CONV), Global Navigation Satellite System (GNSS) Zenith Total Delay (ZTD), and their combination (CONV + ZTD) were assimilated using a 6-hour rapid-update cycle over a full month (20 June–20 July 2021), and verified against independent surface stations withheld from the assimilation, together with radar and GPM IMERG data. For precipitation, DA improved categorical skill: CONV yielded the largest gains in bias (+37.4 %) and probability of detection (+18.3 %), at the cost of a higher false alarm ratio, while CONV + ZTD gave more moderate but balanced improvements. Absolute-error metrics (RMSE, MAE, SMAPE) changed little and mostly not significantly, indicating that DA improves event detection rather than magnitude. For temperature, all configurations reduced bias, ZTD being the most effective (+97.4 %). Station-level Wilcoxon signed-rank tests confirm that the bias and detection gains are statistically significant (p < 0.001). The results demonstrate the complementary roles of conventional and GNSS-based observations for regional numerical weather prediction.
- Preprint
(38252 KB) - Metadata XML
- BibTeX
- EndNote
Status: open (until 10 Sep 2026)
- RC1: 'Comment on egusphere-2026-4222', Anonymous Referee #1, 15 Aug 2026 reply
-
RC2: 'Comment on egusphere-2026-4222', Anonymous Referee #2, 19 Aug 2026
reply
Review of the paper
Title: Evaluating the performance of a numerical weather prediction model for precipitation and temperature in Luxembourg and the Greater Region: insights from WRF and WRFDA 3D-VAR
Author(s): Haseeb ur Rehman et al.
MS No.: egusphere-2026-4222
MS type: Research article
Special issue: Early Warning Systems from Research to Operations: Status, Innovations and Multi-Hazard Applications
General comment
This paper evaluates the performance of the WRF model for precipitation forecast in Luxemburg and Greater Region during the July 2021 flood event. The performance of the model is evaluated with and without data assimilation of conventional and GNSS-ZTD observations. The model performance is evaluated against independent surface stations, mainly located in Luxembourg.
The subject of the paper is interesting and appropriate for NHESS journal. There are, however, several weaknesses that prevent the publication of the paper the current form. Specifically, the potential of the paper is underused for data assimilation, evaluation of model performance and qualitative comparison with observations (nothing is shown for temperature, for example). Moreover, I warmly suggest the use of the CV5 options for DA. This would make the paper more interesting for readers.
Major points
There are some major points that the authors need to address to improve the quality of this paper:
- Simulation set-up: it is not clear, until the Conclusion section that the WRF model was operated in cycling mode for the whole month, with one simulation covering the whole period (if I correctly understood). I suggest to clarify the setting of the model run before, when discussing Figure 4. Also, Figure 4 must be improved to clarify how the model was run.
- Data assimilation: the experiment uses the CV3 option. This option uses a climatological background error matrix. This option reduces the impact of DA on the forecast because it does not account for regionalization (as stated in line 357), which is important for the meso- scale targeted in this paper. I suggest that the authors perform an additional simulation using the CV5 option (and using the NMC method for computing the background error matrix) to improve the impact of data assimilation. This would also give an important sensitivity test to be discussed in the paper.
- Data assimilation: it would be interesting to see statistics of humidity and temperature before and after DA for the different configurations (O-B and O-A analyses) in order to evaluate the impact of different observations in the DA process.
- Figures: Figure 5, showing the interactions of the different WRF physical parameterisation schemes is unnecessary. Figure 4 must be improved to charisma the approach used for simulations. Importantly Figure 12, showing the comparison of the WRF forecasts with IMERG dataset is redundant because the comparison with RADAR wad already shown in Figure 11 for the same times. No comparison are shown for temperature, which is improved by DA (Lines 321-325). Why don’t use Figure 12 to show results for temperature.
- Conclusions: the conclusions are a mere report of the statistics of the paper, which is good. Nevertheless, no comparison is done with the results of other published papers. I suggest to do some comparison with published works to put the results of this paper in a wider context.
Minor points
There are several minor points to see/correct in the paper. I put them as sticky notes in the attached pdf file of the paper. Go through the sticky notes to see/correct these minor points.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 123 | 30 | 16 | 169 | 13 | 18 |
- HTML: 123
- PDF: 30
- XML: 16
- Total: 169
- BibTeX: 13
- EndNote: 18
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
The manuscript addresses a relevant and practically useful question: how much added value do conventional observations and GNSS ZTD bring to WRF forecasts of precipitation and near-surface temperature over Luxembourg and the Greater Region, and does the July 2021 flood episode benefit from assimilation. The experimental design has real strengths. Four experiments (No DA, CONV, ZTD, CONV+ZTD) cycled continuously over a full month is more demanding than the single-case studies that dominate the ZTD assimilation literature, and the authors are right to emphasise this. The use of validation stations withheld from assimilation, the addition of station-level Wilcoxon tests, and the honest reporting of a FAR degradation and of non-significant RMSE changes are all commendable. The writing in the newer passages is clear and self-critical.
However, in its present form the manuscript is not yet ready for publication given reasons. There is no proper Structure of the manuscript, central verification result is under-diagnosed, GNSS ZTD component, which is the novelty claimed in the title and abstract, is never verified in its own observation space and Internal inconsistencies.
Specific Comments:
1. Please restructure manuscript in given form Introduction → Data → Model and Experimental Design → Verification Methodology (current Sect. 2) → Results → Discussion (a genuine discussion section) → Conclusions
2. POD rises from 0.65 (No DA) to 0.78 (CONV), 0.71 (ZTD) and 0.70 (CONV+ZTD), while FAR rises from 0.39 to 0.51, 0.49 and 0.47 respectively. A uniform increase in forecast rain frequency will produce exactly this pattern with no change in discrimination. Please add, for the same 0.1 mm threshold and for at least two higher thresholds (e.g. 1, 5, 10 mm/6 h) e.g. Frequency bias, ETS/CSI, etc
3. Independence of the validation observations is asserted but not demonstrated. match the assimilated and validation station lists by WMO ID and coordinates, report the overlap explicitly, and blacklist any overlapping stations from the assimilation before re-running the affected cycles OR if re-running is not feasible, report the overlap honestly and restrict the "independent verification" claim to the subset of stations that are genuinely withheld.
4. Please state explicitly whether a lapse-rate correction was applied to T2m before verification, apply one (a standard 6.5 K/km, or a model-derived lapse rate), report the distribution of station-model height differences, and repeat the temperature statistics. Do the same for surface pressure if it is used quantitatively.
5. The paper never shows what ZTD assimilation did to the model. Please add (1) OMB and OMA statistics (mean, standard deviation, and number of assimilated observations after QC) for ZTD, and separately for SYNOP, TEMP and TAMDAR, aggregated over the month and stratified by cycle. (2) At least one figure of the analysis increment (e.g. column IWV, 700 hPa specific humidity, T2m) for a representative cycle during 14-15 July, for CONV, ZTD and CONV+ZTD.
6. Section 3 is titled "Results and Discussions" but does not compare the present results with the studies cited in the Introduction. Torcasio et al. (2023), Giannaros et al. (2020), Poletti et al. (2019) and Vladimirov et al. (2020) are described at L63-L75 (P5) and never revisited. Please add a genuine Discussion that addresses at least: How the magnitude of the ZTD impact found here compares with those studies, and why it might be smaller or larger (network density, resolution, background error option, verification metric). Why 12 km grid spacing with parameterised convection (Grell-Freitas, Table 3, P7) limits what can be expected for a convective flood event, and how this interacts with the double-penalty problem when verifying against point gauges. Why the GFS analyses used for IC/BC already contain most of the SYNOP/TEMP/aircraft information being assimilated, which reduces the achievable CONV increment. The physical pathway from a ZTD (moisture) increment to a T2m bias reduction, given that 3D-Var with CV3 does not update soil moisture. The known limitations of CV3 for this application, which the authors already acknowledge well at L164-L167 (P11).
Minor Comments:
L1-L12, P1: The abstract reports percentage improvements without units or baselines. Add the absolute values (for example "bias reduced from -2.27 to -1.42 mm/6 h").
L6-L7, P1: "CONV yielded the largest gains in bias (+37.4 %)" reads as though bias increased. Rephrase as "reduced the precipitation bias by 37.4 %".
L9-L10, P1: "+97.4 %" for temperature bias should be accompanied by the absolute values (-0.39 K to +0.01 K), which makes clear the change is 0.4 K.
L21, P1: missing space in "time(Hapuarachchi et al., 2011)"; missing full stop after the citation.
L23-L24, P2: "Moreover, The likelihood" should be lower case "the"; missing full stop after "(Tom et al., 2022)".
L31, P2: missing full stop after "(Ferreira et al., 2022)".
L34, P2: missing space in "et al., 2023).The".
L40-L41, P2: missing space in "infrastructure(Koks et al., 2022)".
L44, P2: "nearly 253 fatalities" - either "253 fatalities" or "nearly 250".
L48-L49, P2: missing space in "Services(Luxembourg Institute...".
L119-L121, P6: Figures 4 (WRF flow chart) and 5 (physics interaction schematic) are generic and add nothing specific to this study. I recommend removing both or moving them to a supplement. Figure 7 (pre-processing flow chart) is more defensible but could also be simplified.
L121, P6: "shown in Table 4 and Table 3" cites the tables out of order; also Table 3 and Table 4 appear before they are relevant. Renumber so that tables appear in citation order.
L140, P10, Eq. (1): the 3D-Var cost function conventionally carries a factor 1/2 on both terms. As written, J(x) is twice the standard cost function. Please correct or state the convention. Also, the observation error covariance is conventionally denoted R, not O; O is easily confused with the null matrix. Please use R throughout and define all symbols in a consistent notation block.
L205-L208, P12: the observation counts are given as totals with a note about temporal variability. Please instead give the mean number of observations assimilated per cycle for each type, plus the number passing QC, ideally as a time series or a table.
L221-L222, P12/P14: "Individual epochs whose reported uncertainty was anomalously large were discarded" - specify the threshold.
L205-L208, P12: the observation counts are given as totals with a note about temporal variability. Please instead give the mean number of observations assimilated per cycle for each type, plus the number passing QC, ideally as a time series or a table.
L221-L222, P12/P14: "Individual epochs whose reported uncertainty was anomalously large were discarded" - specify the threshold.
L321-L325, P24: the temperature paragraph omits that CONV+ZTD degrades RMSE (2.50 versus 2.41 K, Fig. 14). Negative results should be reported in the text, not only in the figure.
L356-L361, P25: the outlook is sensible. Consider adding assimilation of GNSS slant delays or tomography, hourly cycling, and soil moisture initialisation, given that the temperature bias result points to a surface-layer pathway.
Table 1 and Table 2 are never cited in the text and appear in reverse order. Cite them or remove them; if they are retained, they should be cited in the event-description paragraph around L46-L57 (P2).
Table 2 (P3) reports a 13-year return period for the 2021 annual value of 134.2 mm and ">>100 years" for the summer value, while the text at L51-L53 (P2) quotes 92 years (summer) and 43 years (year) for the daily value and "over 100 years" for the 7-day value. The tables and the text discuss different accumulation periods and are easy to confuse. Please clarify in both captions and text.
Figure 2 (P4) is never cited in the text.
Figure 3 (P8): the caption states that plus signs mark stations plotted "in Figures 9, 8, and 10" - reorder. Also, the figure should distinguish assimilated stations from validation stations, ideally with different symbols.