An objective dynamic multivariable weighting method for reducing uncertainty in WRF parameterization scheme selection

Gou, Tianyu; Deng, Yaoyang; Niu, Jun; Kang, Shaozhong

doi:10.5194/egusphere-2025-5362

Preprints

https://doi.org/10.5194/egusphere-2025-5362

Preprints

19 Dec 2025

| 19 Dec 2025

An objective dynamic multivariable weighting method for reducing uncertainty in WRF parameterization scheme selection

Tianyu Gou, Yaoyang Deng, Jun Niu, and Shaozhong Kang

Abstract. The selection of optimal physical parameterization schemes is a major source of uncertainty in WRF model simulations. A comprehensive evaluation of model performance requires simultaneous consideration of multiple output variables. However, existing multivariate approaches often rely on subjective or overly simplistic equal-weighting strategies and lack an objective mechanism to quantify variable importance. Such limitations can obscure significant biases in poorly simulated variables. To overcome this issue, this study proposes an objective dynamic weighting method for multivariate evaluation. The method employs a two-layer weighting framework based on two statistical metrics: the mean relative error, which measures the simulation accuracy of a variable, and the coefficient of variation of the absolute error, which reflects the sensitivity of a variable to the physical process under evaluation. The approach is applied and validated in assessing WRF parameterization schemes across two climatically distinct environments: the arid Northwest and the humid coastal Southeast of China. The results show that the method assigns greater weights to poorly simulated variables, such as precipitation and wind speed, thereby enabling the identification of more physically plausible and robust parameterization scheme combinations. Compared with the equal-weighting method, the scheme combinations obtained using this approach produce a lower Multivariate Integrated Evaluation Index (MIEI), a higher correlation coefficient (R), a lower Root Mean Square Error (RMSE), and exhibit superior performance in independent extreme-year validations. By dynamically incorporating both simulation performance and sensitivity specific to each variable, the method offers a more rigorous and objective framework for model evaluation and uncertainty reduction.

Received: 29 Oct 2025 – Discussion started: 19 Dec 2025

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors. Views expressed in the text are those of the authors and do not necessarily reflect the views of the publisher.

Download & links

Preprint (PDF, 1767 KB)

Supplement (182 KB)

Download & links

Tianyu Gou, Yaoyang Deng, Jun Niu, and Shaozhong Kang

Status: final response (author comments only)

RC1:
'Comment on egusphere-2025-5362', Anonymous Referee #1, 16 Feb 2026
The authors present a novel framework for selecting physical parameterization schemes within the Weather Research and Forecasting (WRF) model. The study proposes a "dynamic weighting" approach that assigns importance to output variables based on two statistical criteria: simulation accuracy (Mean Relative Error) and sensitivity to physical processes (Coefficient of Variation of the absolute error). This method is applied to two climatically distinct regions in China and compared against a traditional equal-weighting strategy. The authors further validate their approach using an independent extreme weather year.
The manuscript is generally well-structured and addresses a relevant issue in regional climate modeling: the subjectivity involved in evaluating multivariate model performance. The proposed tool offers a potential pathway toward more reliable, data-driven parameterization selection and uncertainty reduction. However, the methodology requires further justification regarding the mathematical formulation of the weights, the sequential selection methodology, and the physical interpretation of the results. I recommend a moderate revision before possible publication in GMD, addressing these points detailed below.
General Comments
The authors employ the Softmax function to normalize statistical metrics into weights, a mathematical choice that inherently tends to polarize values. While the authors state this is to "amplify the impact of key limiting variables," this exponential scaling risks excessively penalizing variables with only marginally higher errors or sensitivities, potentially reducing a multivariate evaluation to a quasi-univariate one dominated by a single variable like precipitation. To establish the scientific robustness of this approach, it is essential to provide a justification for selecting Softmax over linear normalization techniques and to include a sensitivity analysis to ensure the method does not artificially skew the evaluation.

The study selects specific individual years to represent the climatological mean based on 30-year averages. However, relying on a single year for parameterization optimization may inadvertently overfit the model to specific synoptic patterns unique to that year, rather than capturing the region's broader climatological variability.The authors need to provide detailed assessment methods and quantitative analysis to prove that the selected year is statistically representative across all assessment variables, rather than merely representing the average state. Furthermore, a discussion regarding the potential biases introduced by single-year optimization versus multi-year validation is necessary to contextualize the universality of the selected schemes.

The authors employed an ordered selection strategy. As discussed in the relevant section (lines 571-579), this approach inevitably introduces path dependence.The optimal choice for the initial process inevitably constrains the performance landscape for subsequent choices, suggesting that the final combination likely represents a local optimum dependent on the arbitrary order of evaluation rather than a global optimum. The authors should critically discuss the theoretical implications of this limitation and, if possible, justify the specific order chosen.

While the manuscript presents comprehensive statistical evidence, it lacks sufficient analysis of the underlying physical mechanisms driving these results. The authors are encouraged to discuss the physical mechanisms of the optimal parameterization schemes by linking the method's characteristics with the regional climatic features. This will enhance the theoretical depth of the paper.

The discussion suggests that the dynamic weighting method is transferable to multi-model evaluations. However, weighting distinct models with structural differences presents different challenges compared to selecting parameterization schemes within the unified WRF framework. The authors should expand this section to clarify how the methodology would be adapted for multi-model ensembles, specifically addressing how it accounts for structural uncertainties versus parameter perturbations.

Specific Comments
P1, L13: "an objective dynamic weighting method" — I advise caution with the word "objective." The choice of the Softmax function, the choice of the variable set (7 variables), and the choice of the evaluation order are all subjective decisions made by the researchers. "Data-driven" or "Performance-based" might be more accurate terms.

P2, L43: "To evaluate the overall performance of the model (Gleckler et al., 2008), proposed..." should be corrected to "To evaluate the overall performance of the model, Gleckler et al. (2008) proposed...".

P2, L64: "total of over 14.5 million possible combinations" — While mathematically true, many of these are physically invalid or incompatible. It would be better to cite a realistic number of feasible combinations if possible, or acknowledge this is a theoretical upper bound.

P4, L114: "we will conduct" should be changed to "we conducted" or "we conduct" to align with standard academic tense for methodology.

P5, L140: "Northwest of China" should be corrected to "Northwest China" for consistency.

P7, L197: "suffering from severe flooding" should be changed to "experiencing severe flooding" or "affected by severe flooding" to maintain an objective academic tone.

P13, L324: "assigned on the basis of" is wordy and should be condensed to "assigned based on" to improve flow.

P24, L565: "simplifies the complex multiplicative combinations... into additive combinations." This phrasing is technically incorrect. The physical processes are coupled non-linearly (multiplicative nature). The step-wise selection method treats them sequentially (additive nature of the process, not the physics). Please rephrase to avoid implying that the method changes the physical nature of the interactions.

P25, L598: The sentence "...yields more accurate temperature and wind speed, the dynamic weighting scheme provides more accurate results..."The two clauses are tautological/redundant, making the sentence logically circular.

Figure 2 and Figure 5: The x-axis represents discrete categories (Microphysics, PBL, etc.), yet the values are connected by lines. This implies continuity which does not exist. A bar chart or a scatter plot would be a more appropriate visualization for these weights to avoid misleading the reader.

Figure 3(a) and Figure 6(a): The color scale for MIEI seems to show "normalized" values. It would be helpful to clarify in the caption what the normalization base is (e.g., normalized by the median of the ensemble?).
Citation: https://doi.org/10.5194/egusphere-2025-5362-RC1
RC2: 'Comment on egusphere-2025-5362', Anonymous Referee #2, 27 May 2026

An objective dynamic multivariable weighting method for reducing uncertainty in WRF parameterization scheme selection

The manuscript addresses a relevant problem in WRF parameterization scheme selection, namely how to evaluate and compare different parameterization combinations using multiple meteorological variables. However, the main methodological contribution is not yet convincing. The proposed “dynamic” weighting method appears to be a post hoc error-based weighting scheme, but the manuscript does not sufficiently demonstrate its novelty, theoretical basis, robustness, or advantage over existing objective weighting methods. Major methodological clarification and additional validation are therefore needed. My detailed comments are as follows:

1. The comparison with equal weighting alone is too weak to establish methodological superiority. Weighting variables according to error magnitude or error dispersion is a common idea in multivariable evaluation, and the manuscript should demonstrate how the proposed scheme differs conceptually and quantitatively from established objective weighting approaches (e.g., entropy weighting, coefficient-of-variation weighting, CRITIC-type methods, and other sensitivity-based evaluation frameworks).

2. The meaning of “dynamic” is unclear. The weights seem to be calculated after the simulations are completed, rather than being dynamically updated during model simulation or evaluation.

3. The objectivity of the method is overstated. Although the weights are calculated from data, several key components of the method are still based on subjective methodological choices, including the selection of MRE and CV as the two weighting metrics, the use of the Softmax transformation, and the multiplicative combination of the two weight components. These choices require stronger theoretical or empirical justification before the method can be presented as an objective weighting framework.

4. The manuscript does not consider correlations among the evaluated variables. The authors should discuss how inter-variable dependence may affect the multivariable evaluation and whether any decorrelation or redundancy-control procedure is needed.

5. The stability of the derived weights is not adequately examined. It is unclear whether the variable weights would remain similar if a different year, station subset, time period, or variable set were used. Without such stability tests, the derived weights may reflect sampling noise rather than robust variable importance.

6. The final weight definition requires further clarification and justification. The product of two Softmax-normalized weights does not generally sum to one, so an additional normalization step should be specified if the product is used as the final variable weight. Moreover, the multiplicative structure gives high final weights mainly to variables that are high in both components. The authors should justify this choice and examine whether the selected parameterization schemes are robust to alternative combination rules (e.g., weighted additive formulation).

7. The Introduction is overly lengthy and lacks a clear focus.

8. The manuscript should more clearly distinguish between model evaluation and model improvement. The proposed weighting method changes how different variables contribute to the evaluation index and may help identify better-performing scheme combinations, but it does not directly improve the WRF simulations themselves. Therefore, statements suggesting that the method “improves the model's simulations” (line 599) should be revised.

9. The temporal aggregation used to calculate the evaluation metrics is unclear. The manuscript should specify whether R, RMSE, MIEI, and the Taylor diagrams are based on daily values, monthly means, seasonal means, or climatological averages, because these choices can lead to substantially different conclusions.

10. The results do not clearly explain how the dynamic weights change the ranking of parameterization schemes compared with equal weighting. The authors should show which variables receive higher weights and how these weights directly affect the selection of the final optimal schemes.

11. The Taylor diagrams show that many parameterization schemes are clustered closely together, and the improvement of the optimal scheme appears visually small. The authors should provide statistical significance tests, confidence intervals, bootstrap resampling, or cross-validation analyses to show whether the differences between the selected schemes exceed the uncertainty arising from temporal variability, station sampling, and observational errors.

Citation: https://doi.org/10.5194/egusphere-2025-5362-RC2
RC3:
'Comment on egusphere-2025-5362', Anonymous Referee #3, 04 Jun 2026
General Comments
In general, I consider that the manuscript addresses a relevant problem for the WRF modeling community, since the selection of physical parameterization schemes remains one of the main sources of uncertainty in regional atmospheric simulations. The proposed weighting strategy based on the performance of multiple variables is interesting and has the potential to contribute to a more comprehensive evaluation of model configurations.
However, I believe that some aspects could be strengthened to increase the scientific value of the study.
The manuscript would benefit from a deeper discussion of the physical interpretation of the selected parameterization schemes. Although the methodology successfully identifies the configurations that provide the best statistical performance, the discussion remains focused mainly on the evaluation metrics. It would be valuable to explain why the selected schemes may be physically suitable for the climatic and atmospheric characteristics of the analyzed regions. Such discussion would help readers understand whether the methodology is identifying parameterizations that are not only statistically optimal but also physically consistent with the dominant atmospheric processes.

The uncertainty analyzed in this study is associated with the selection of physical parameterization schemes. However, parameterization choice represents only one component of the overall uncertainty present in WRF simulations. Other factors, such as spatial resolution, domain configuration, land-use representation, topography, initial and boundary conditions, and spin-up time may also influence model performance. A brief discussion clarifying the specific scope of the proposed methodology and its relationship to other sources of uncertainty would strengthen the interpretation of the results.

The manuscript demonstrates differences among the evaluated parameterization combinations; however, it would be useful to discuss the practical implications of these differences from a meteorological perspective. In several cases, the statistical differences between competing configurations appear relatively small. Therefore, it would be interesting to discuss whether these improvements translate into a substantially better representation of the atmospheric processes of interest or whether they mainly reflect differences in the evaluation metrics. This would help readers better assess the practical significance of the proposed methodology.

Overall, I believe that the manuscript addresses an important topic and presents an interesting methodological contribution. Expanding the physical interpretation of the results and clarifying the scope of the uncertainty analysis would further strengthen the paper and improve its relevance for researchers and practitioners working with WRF.
Citation: https://doi.org/10.5194/egusphere-2025-5362-RC3

Tianyu Gou, Yaoyang Deng, Jun Niu, and Shaozhong Kang

Supplement

https://doi.org/10.5194/egusphere-2025-5362-supplement

Model code and software

An objective dynamic multivariable weighting method for reducing uncertainty in WRF parameterization scheme selection Tianyu Gou et al. https://doi.org/10.5281/zenodo.17414002

A Description of the Advanced Research WRF Model Version 4.3 W. Skamarock et al. https://doi.org/10.5065/1dfh-6p97

Tianyu Gou, Yaoyang Deng, Jun Niu, and Shaozhong Kang

Viewed

Total article views: 1,939 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	Supplement	BibTeX	EndNote
1,169	579	191	1,939	191	233	231

HTML: 1,169
PDF: 579
XML: 191
Total: 1,939
Supplement: 191
BibTeX: 233
EndNote: 231

Views and downloads (calculated since 19 Dec 2025)

Month	HTML	PDF	XML	Total
Dec 2025	390	175	40	605
Jan 2026	308	170	45	523
Feb 2026	146	73	21	240
Mar 2026	228	114	76	418
Apr 2026	51	17	5	73
May 2026	39	25	1	65
Jun 2026	7	5	3	15

Cumulative views and downloads (calculated since 19 Dec 2025)

Month	HTML	PDF	XML	Total
Dec 2025	390	175	40	605
Jan 2026	308	170	45	523
Feb 2026	146	73	21	240
Mar 2026	228	114	76	418
Apr 2026	51	17	5	73
May 2026	39	25	1	65
Jun 2026	7	5	3	15

Viewed (geographical distribution)

Total article views: 1,931 (including HTML, PDF, and XML) Thereof 1,931 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 04 Jun 2026

Short summary

This study proposes a new method to improve climate simulation evaluation, tackling a key model error: selecting the best parameter combinations. Our "dynamic weighting" method automatically gives more importance to hard-to-predict variables, like precipitation and wind speed. When tested in two distinct climate regions, our approach identified model settings that produced more accurate and reliable forecasts than traditional equal-weighting methods, performing well in extreme weather years.


Total:	0
HTML:	0
PDF:	0
XML:	0