the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 22 Nov 2024
Benchmarking historical performance and future projections from a global hydrologic model with a basin-scale model
Abstract. Global hydrologic models (GHMs) are increasingly relied upon for assessing climate-driven hydrologic changes from watershed to global scales. However, their ability to provide robust projections for a range of hydrologic variables remains unclear. Here, we evaluate the historical performance and future projections from the Community Water Model (CWatM) GHM against the Variable Infiltration Capacity (VIC) watershed hydrologic model for the Liard River basin in subarctic Canada. We drive both models with an ensemble of eight global climate models from the Coupled Model Intercomparison Project phase 6, downscaled and bias-corrected with a multivariate method. We analyze a range of hydrologic projections at 1.5 to 4.0 °C global warming levels (GWLs) above the preindustrial period. The historical performance benchmarking shows reasonable goodness-of-fit metrics for both models, with a slightly better performance for VIC. Projected hydrologic responses from CWatM are generally consistent with VIC in terms of annual water balance, and monthly snow water equivalent and flow changes, suggesting the robustness of the projections. Both models project coherent hydrologic changes, including progressively higher annual evapotranspiration; increased annual, winter, spring and maximum flows; increased frequency of extreme flow; and earlier timing of maximum flow, with higher GWLs. However, the magnitudes of maximum flow and late summer flow diverge between the two models, which can be explained by structural uncertainties associated with the representation of frozen soil and groundwater processes. Thus, our study provides insights into the robustness of hydrologic projections from a GHM, and offers a basis for model improvements.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(2325 KB)
-
Supplement
(671 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2325 KB) - Metadata XML
-
Supplement
(671 KB) - BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-3170', Anonymous Referee #1, 06 Jan 2025
The study by Shrestha et al. presents a benchmarking analysis of a global hydrological model (CWATM) against a watershed hydrological model (VIC) for future projections. The paper is well-structured, clearly written, and achieves its stated goals effectively. The comparative approach appears robust and provides valuable insights into the models' performances and I especially appreciate that uncertainties are being clearly discussed.
The study would benefit from even more emphasising how the models' water balance components compare to real-world observations. In Section 4.4, adding comparisons to observed data for key water balance components such as annual (monthly) streamflow and the distribution of precipitation into evaporation and runoff, would help contextualize each models’ performance against real-world observations and each other.
Additionally, while the abstract highlights the identification of areas for model improvement (l. 22-23) and the introduction points towards limitations of these models, the potential areas for improvement are not as clearly outlined in the conclusion. To do this, however, could enhance the study's findings and clarity.Technical comments
- Improve the placement of Tables 1, 2, and 3 and Figure 6 to enhance clarity and avoid confusion. The items are not exactly located where they are mentioned in the text, which I found confusing at times.
- Clarify in Table 1 that Manning’s n refers to a roughness coefficient.
- Provide a complete caption for Figure 6 to ensure it can stand independently, even if it is similar to Figure 5.
- Consider providing consistent ‘units’ in Table 1, since the table features arc-minutes, arc-seconds and degrees for grid-scale size references. This makes it difficult to get a good idea of the actual differences in scale.
Citation: https://doi.org/10.5194/egusphere-2024-3170-RC1 -
AC1: 'Reply on RC1', Rajesh Shrestha, 26 Feb 2025
Response to Anonymous Referee #1 (shown in bold and italic)
The study by Shrestha et al. presents a benchmarking analysis of a global hydrological model (CWATM) against a watershed hydrological model (VIC) for future projections. The paper is well-structured, clearly written, and achieves its stated goals effectively. The comparative approach appears robust and provides valuable insights into the models' performances and I especially appreciate that uncertainties are being clearly discussed.
We thank the reviewer for taking time to review our manuscript and positive comments. We will carefully address all comments in the revised manuscript. At this stage, we would like to focus our response on the main points raised by the reviewer.
The study would benefit from even more emphasising how the models' water balance components compare to real-world observations. In Section 4.4, adding comparisons to observed data for key water balance components such as annual (monthly) streamflow and the distribution of precipitation into evaporation and runoff, would help contextualize each models’ performance against real-world observations and each other.
We agree with the reviewer that comparing historical and projected changes with observations will be beneficial to contextualize the results against real world observations. To that end, we will add observed streamflow in Section 4.4 Figure, and discuss the models’ performance against observations. However, evaporation observations are not available to compare with the model results.
Additionally, while the abstract highlights the identification of areas for model improvement (l. 22-23) and the introduction points towards limitations of these models, the potential areas for improvement are not as clearly outlined in the conclusion. To do this, however, could enhance the study's findings and clarity.
We also agree with the reviewer that it will be beneficial to outline potential areas of model improvements. Based on the study findings, we will provide an outline of potential improvements in both CWatM and VIC models in the conclusions section.
Technical comments
- Improve the placement of Tables 1, 2, and 3 and Figure 6 to enhance clarity and avoid confusion. The items are not exactly located where they are mentioned in the text, which I found confusing at times.
- Clarify in Table 1 that Manning’s n refers to a roughness coefficient.
- Provide a complete caption for Figure 6 to ensure it can stand independently, even if it is similar to Figure 5.
- Consider providing consistent ‘units’ in Table 1, since the table features arc-minutes, arc-seconds and degrees for grid-scale size references. This makes it difficult to get a good idea of the actual differences in scale.
We will make all suggested technical changes in the revised manuscript.
Citation: https://doi.org/10.5194/egusphere-2024-3170-AC1
-
RC2: 'Comment on egusphere-2024-3170', Anonymous Referee #2, 23 Jan 2025
I enjoyed reviewing the manuscript "Benchmarking historical performance and future projections from a global hydrologic model with a basin-scale model" by Rajesh R. Shrestha and co-authors and I hope that my comments will help to further improve it. In this study, the authors compare simulations and predictions of two hydrological models (the Community Water Model, CWatM, and Variable Infiltration Capacity, VIC), both calibrated for the Canadian Liard River basin. The authors found satisfactory model performances of both models when simulating a historical time period. When using the models to predict changes under different future scenarios, the results of both models were mainly consistent. The manuscript is understandable and interesting, as well as clearly and concisely written.
General comments
In general, I think a clarification on the definition of a global hydrologic model (GHM) is required. In the manuscript, the CWatM is considered as a global model that is compared to the VIC, a watershed model (WHM). However, according to the model development paper (Burek et al., 2020), the CWatM is not solely a global, but also a regional model. This is also stated in Sect. 3.1, lines 109-110. Since here, the CWatM is calibrated to one specific basin, one could argue that it is used as a regional model and not as a global model. In fact, the main difference between the application of the CWatM and the VIC seems to be the number of subcatchments for which the models were calibrated. This is still a valuable comparison, but needs to be framed differently (i.e., not as a comparison of a GHM to a WHM) in my opinion. I recommend to either explain more clearly why the CWatM can still be considered a GHM in this application case, or to adapt the goal of the manuscript accordingly. Related to this, it may be interesting to also include how well a model performs (or, how well the two models perform) for this basin when not specifically calibrated as a comparison, i.e., as a counterpart to the benchmark of the calibrated WHM.
Furthermore, I see a general challenge in how the calibration process was designed: According to Table 1, the calibration of the VIC takes the NSE, LNSE, and VB into account, while the CWatM is calibrated on the KGE. In my opinion, both models should be treated analogously during the calibration process, i.e., calibrated on the same objective function. Currently, the different criteria during calibration lead to additional uncertainties regarding the differences in model performance that can be avoided when the same criteria are used. Except if there is a reason for the different treatment of the two models (that should then be stated clearly, and the effects on the results should be included in the discussion), one of the model calibrations needs to be repeated with the objective function that was used for the other model. All analyses should then be based on those results.
Specific comments
- In lines 115-116, consider including the paper by Hanus et al. (2024, 10.5194/gmd-17-5123-2024) on coupling the CWatM with a glacier model as an example how the CWatM should become more feasible for the application in cold regions.
- Between Sect. 3.2 and Sect. 3.3, I am missing some information on how the models were calibrated. This information is available in Table 1, but it would be helpful to at least have a remark on where to find this information between the explanation on the forcing data and the evaluation methods.
- When providing the reader with the model performance results (currently Table 3), it will be important to indicate which of the performance measures were used in the calibration process, i.e., which of the measures were optimized (see general comment on the different objective functions).
- Please include the black square in the legend in Figure 3. Please also add in the caption of Figure 3 what the different datapoints per column, colour, and symbol (e.g., the different purple triangles in the same column) refer to. Furthermore, in Figure 3 consider to: not capitalize “temperature” and “precipitation”, add a space before “°C” in the legend, and write the name of the SSPs consistently in the text (lines 155 and 196), in the caption and in the legend.
- Please include in the caption of Figure 5 that you did not include the summer months on purpose (I assume because they are not interesting regarding the SWE).
- Currently, Sect. 4 is entitled “Results and discussion” and Sect. 5 is entitled “Discussion and Conclusions”. Please make sure that you clearly indicate where your results are discussed. I suggest you to follow the common standard of first presenting your results (Sect. 4) and then discussing them (Sect. 5), i.e., to move all discussion parts currently included in Sect. 4 to Sect. 5 and thus have only one comprehensive discussion. Concluding remarks could be added to Sect. 5 (“Discussion and conclusions”) or as a separate Sect. 6.
- The study is based on one basin only. This has the advantage that detailed analyses are possible, and different processes (as opposed to only model performances regarding streamflow simulation) can be considered. This is nicely done in the manuscript. However, thanks to the growing computational power, model comparison studies can nowadays rather easily be based on a larger number of catchments. Such studies have the advantage that the results are more robust and can be generalized, as the variability between different catchments can be obtained. In a study with only one catchment, this is not possible. Please add this in the discussion, i.e., make sure to emphasize that it is likely possible that the results are different in other application cases.
Technical suggestions
- Consider including axis labels, a north arrow, and a scale bar in Figure 1.
- For consistency, in the title of Sect. 3 “Models, Data and Analyses”, the terms “data” and “analyses” should not be capitalized.
- I agree with the first reviewer that a common unit for the different entries in Table 1 would make comparison easier. I suggest to add this as a complement to the units that are given in this table now (i.e., also keep the units that are given with each dataset).
- In the column “spatial resolution”, for VIC, the word “for” before “snow” seems to be missing.
- In Sect. 1 and Sect. 4.1, for example, you use “x” as a sign for multiplication, in Sect. 4.5 you use “*”. Consider changing this for consistency.
- For all occurrences of the colour-bar legends (median change and model agreement), consider replacing the term “model agreement” with “GCM agreement” to avoid confusion with the agreement of the two models VIC and CWatM.
- The Oxford comma is not used consistently, this could be improved.
- Consider including the full term in addition to the abbreviation when using an abbreviation in a Figure or Table caption (as you did it for example for “GWL” in the caption of Figure 4).
- Considering the facts that there are already many abbreviations in the manuscript and that you do not make use of the abbreviation “LRB” very often, I recommend that you do not use an abbreviation for the Liard River basin at all. Similarly, for “GEV” abbreviating “generalized extreme value”: After introducing this abbreviation, it is not used anymore, so I would leave it away.
- You sometimes use “()” around units in Figures and sometimes “[]”, this may be improved for consistency. Similarly, you should use “s” instead of “sec” as an abbreviation of “second” and potentially also decide for either “discharge” or “flow” in the different plots showing discharge or flow values.
Citation: https://doi.org/10.5194/egusphere-2024-3170-RC2 -
AC2: 'Reply on RC2', Rajesh Shrestha, 26 Feb 2025
Response to Anonymous Referee #2 (shown in bold and italic)
I enjoyed reviewing the manuscript "Benchmarking historical performance and future projections from a global hydrologic model with a basin-scale model" by Rajesh R. Shrestha and co-authors and I hope that my comments will help to further improve it. In this study, the authors compare simulations and predictions of two hydrological models (the Community Water Model, CWatM, and Variable Infiltration Capacity, VIC), both calibrated for the Canadian Liard River basin. The authors found satisfactory model performances of both models when simulating a historical time period. When using the models to predict changes under different future scenarios, the results of both models were mainly consistent. The manuscript is understandable and interesting, as well as clearly and concisely written.
We thank the reviewer for taking time to review our manuscript and constructive comments. We will carefully address all comments in the revised manuscript. At this stage, we would like to focus our response on the main points raised by the reviewer.
General comments
In general, I think a clarification on the definition of a global hydrologic model (GHM) is required. In the manuscript, the CWatM is considered as a global model that is compared to the VIC, a watershed model (WHM). However, according to the model development paper (Burek et al., 2020), the CWatM is not solely a global, but also a regional model. This is also stated in Sect. 3.1, lines 109-110. Since here, the CWatM is calibrated to one specific basin, one could argue that it is used as a regional model and not as a global model. In fact, the main difference between the application of the CWatM and the VIC seems to be the number of subcatchments for which the models were calibrated. This is still a valuable comparison, but needs to be framed differently (i.e., not as a comparison of a GHM to a WHM) in my opinion. I recommend to either explain more clearly why the CWatM can still be considered a GHM in this application case, or to adapt the goal of the manuscript accordingly.
Based on the reviewer’s comment, we feel that it is important to distinguish the model application from the model itself when describing the global hydrologic model (GHM). As the reviewer pointed out, CWatM is designed for global and regional assessments (Burek et al., 2020), however, it not tailored to a specific region or a basin. Furthermore, it can simulate terrestrial water cycle processes and human usage on a global scale due to the incorporation of global scale datasets (Burek et al., 2020; Telteu et al., 2021). CWatM has also been used in several global scale assessments as a part of a multi-GHM ensemble (e.g. Boulange et al., 2021; Pokhrel et al., 2021; Satoh et al., 2022). Due to these reasons, we feel that CWatM should classified as a GHM. However, we agree with the reviewer that our application is actually a basin scale. In fact, the main objective of our study is to assess the capability of a GHM – not tailored to represent basin scale processes (specifically, cold-regions processes) - in replicating historical simulations and future projections from a watershed hydrologic model. In the revised version, we will reframe our study by adding these clarifications.
Related to this, it may be interesting to also include how well a model performs (or, how well the two models perform) for this basin when not specifically calibrated as a comparison, i.e., as a counterpart to the benchmark of the calibrated WHM.
The reviewer made an interesting suggestion to compare uncalibrated models as an additional benchmarking exercise. However, the performance of an uncalibrated model will depend on the set of uncalibrated parameters used, as the model with one set of uncalibrated parameters could produce totally different results compared to the model with another set of uncalibrated parameters. Due to this reason, we do not think comparing calibrated and uncalibrated model results will provide a fair assessment of the model’s capability.
Furthermore, I see a general challenge in how the calibration process was designed: According to Table 1, the calibration of the VIC takes the NSE, LNSE, and VB into account, while the CWatM is calibrated on the KGE. In my opinion, both models should be treated analogously during the calibration process, i.e., calibrated on the same objective function. Currently, the different criteria during calibration lead to additional uncertainties regarding the differences in model performance that can be avoided when the same criteria are used. Except if there is a reason for the different treatment of the two models (that should then be stated clearly, and the effects on the results should be included in the discussion), one of the model calibrations needs to be repeated with the objective function that was used for the other model. All analyses should then be based on those results.
With regards to the reviewer’s suggestion of using the same objective function for an analogous calibration of both models, we would like to clarify that we used the calibrated VIC model with three objective functions (NSE, LNSE and VB) from our previous study (Shrestha et al., 2019), while CWatM was calibrated by using the KGE objective function (which is a combination of correlation, bias ratio and variability ratio) as designed for a typical calibration setup by the model developers (Burek et al., 2020). While we agree that the use of different objective functions could add some uncertainty, we would like to note that the selection of an objective function is only a part of the challenge for model calibration. More importantly, several different parameter combinations within a chosen model structure could yield similar performance due to model equifinality (Beven, 2006). Given this unavoidable challenge, we took additional steps for selecting the “best parameter sets” from the calibration runs. Specifically, we ran the NSGA-II calibration setups for both models at least three times and considered the models’ ability to simulate peak flow, low flow and water balance by visualizing the hydrographs and annual water balance obtained from top parameter sets from each calibration run. We used the modeller’s judgement for selecting the “best parameter set”, which is not necessarily the parameter set with the best score for one of the objective functions considered. The effect of this additional step can be seen in the performance the two models for the Liard-UC station over the calibration period. Specifically, although the CWatM model for this station was calibrated by using the KGE objective function, the KGE performance for the selected CWatM model parameters is inferior to that of VIC (KGE was not used for VIC calibration), while the NSE and LNSE (used in calibration of VIC but not CWatM) performances of CWatM are similar to VIC. Thus, our calibration can be described as automatic calibration supplemented by the modeller’s judgement. We did not provide these details in the original version of manuscript, with a thinking that model calibration is not the main focus of our study. However, the reviewer has correctly pointed out the need to provide further details on calibration. In the revised version, we will add a paragraph detailing the points outlined above.
Specific comments
- In lines 115-116, consider including the paper by Hanus et al. (2024, 10.5194/gmd-17-5123-2024) on coupling the CWatM with a glacier model as an example how the CWatM should become more feasible for the application in cold regions.
Thank you for the suggestion, we will reference this study as an application of the CWatM model for cold regions.
- Between Sect. 3.2 and Sect. 3.3, I am missing some information on how the models were calibrated. This information is available in Table 1, but it would be helpful to at least have a remark on where to find this information between the explanation on the forcing data and the evaluation methods.
As discussed above, we will add a paragraph outlining our calibration steps.
- When providing the reader with the model performance results (currently Table 3), it will be important to indicate which of the performance measures were used in the calibration process, i.e., which of the measures were optimized (see general comment on the different objective functions).
We agree with the reviewer, and we will indicate the performance metrics used as objective functions in the calibration of each model in Table 3.
- Please include the black square in the legend in Figure 3. Please also add in the caption of Figure 3 what the different datapoints per column, colour, and symbol (e.g., the different purple triangles in the same column) refer to. Furthermore, in Figure 3 consider to: not capitalize “temperature” and “precipitation”, add a space before “°C” in the legend, and write the name of the SSPs consistently in the text (lines 155 and 196), in the caption and in the legend.
We agree with the reviewer, and will make the suggested technical edits in Figure 3 legend and caption.
- Please include in the caption of Figure 5 that you did not include the summer months on purpose (I assume because they are not interesting regarding the SWE).
We will add an explanation in the Figure 5 caption that summer months were not included in the Figure because of the lack of snow.
- Currently, Sect. 4 is entitled “Results and discussion” and Sect. 5 is entitled “Discussion and Conclusions”. Please make sure that you clearly indicate where your results are discussed. I suggest you to follow the common standard of first presenting your results (Sect. 4) and then discussing them (Sect. 5), i.e., to move all discussion parts currently included in Sect. 4 to Sect. 5 and thus have only one comprehensive discussion. Concluding remarks could be added to Sect. 5 (“Discussion and conclusions”) or as a separate Sect. 6.
It was our intention our to provide “Results and Discussion” together in section 4, and “Summary and Conclusions” in section 5, and we structured the sections accordingly. It was an oversight on our part that section 5 was mistakenly entitled “Discussion and Conclusions”. We will use the correct heading for section 5 in the revised version of the manuscript.
- The study is based on one basin only. This has the advantage that detailed analyses are possible, and different processes (as opposed to only model performances regarding streamflow simulation) can be considered. This is nicely done in the manuscript. However, thanks to the growing computational power, model comparison studies can nowadays rather easily be based on a larger number of catchments. Such studies have the advantage that the results are more robust and can be generalized, as the variability between different catchments can be obtained. In a study with only one catchment, this is not possible. Please add this in the discussion, i.e., make sure to emphasize that it is likely possible that the results are different in other application cases.
The reviewer raised an important point regarding generalization of the results, given our study is focused on a single basin. Although our study basin Liard is large (about 275,000 km2), and consistent results for Liard-UC subbasin improve confidence in our results, we agree that broader generalization may not possible. We will discuss this point in the conclusions section.
Technical suggestions
- Consider including axis labels, a north arrow, and a scale bar in Figure 1.
- For consistency, in the title of Sect. 3 “Models, Data and Analyses”, the terms “data” and “analyses” should not be capitalized.
- I agree with the first reviewer that a common unit for the different entries in Table 1 would make comparison easier. I suggest to add this as a complement to the units that are given in this table now (i.e., also keep the units that are given with each dataset).
- In the column “spatial resolution”, for VIC, the word “for” before “snow” seems to be missing.
- In Sect. 1 and Sect. 4.1, for example, you use “x” as a sign for multiplication, in Sect. 4.5 you use “*”. Consider changing this for consistency.
- For all occurrences of the colour-bar legends (median change and model agreement), consider replacing the term “model agreement” with “GCM agreement” to avoid confusion with the agreement of the two models VIC and CWatM.
- The Oxford comma is not used consistently, this could be improved.
- Consider including the full term in addition to the abbreviation when using an abbreviation in a Figure or Table caption (as you did it for example for “GWL” in the caption of Figure 4).
- Considering the facts that there are already many abbreviations in the manuscript and that you do not make use of the abbreviation “LRB” very often, I recommend that you do not use an abbreviation for the Liard River basin at all. Similarly, for “GEV” abbreviating “generalized extreme value”: After introducing this abbreviation, it is not used anymore, so I would leave it away.
- You sometimes use “()” around units in Figures and sometimes “[]”, this may be improved for consistency. Similarly, you should use “s” instead of “sec” as an abbreviation of “second” and potentially also decide for either “discharge” or “flow” in the different plots showing discharge or flow values.
We will address all technical suggestions in the revised manuscript.
References
Beven, K.: A manifesto for the equifinality thesis, Journal of Hydrology, 320, 18–36, https://doi.org/10.1016/j.jhydrol.2005.07.007, 2006.
Boulange, J., Hanasaki, N., Satoh, Y., Yokohata, T., Shiogama, H., Burek, P., Thiery, W., Gerten, D., Schmied, H. M., Wada, Y., Gosling, S. N., Pokhrel, Y., and Wanders, N.: Validity of estimating flood and drought characteristics under equilibrium climates from transient simulations, Environ. Res. Lett., 16, 104028, https://doi.org/10.1088/1748-9326/ac27cc, 2021.
Burek, P., Satoh, Y., Kahil, T., Tang, T., Greve, P., Smilovic, M., Guillaumot, L., Zhao, F., and Wada, Y.: Development of the Community Water Model (CWatM v1.04) – a high-resolution hydrological model for global and regional assessment of integrated water resources management, Geoscientific Model Development, 13, 3267–3298, https://doi.org/10.5194/gmd-13-3267-2020, 2020.
Pokhrel, Y., Felfelani, F., Satoh, Y., Boulange, J., Burek, P., Gädeke, A., Gerten, D., Gosling, S. N., Grillakis, M., Gudmundsson, L., Hanasaki, N., Kim, H., Koutroulis, A., Liu, J., Papadimitriou, L., Schewe, J., Müller Schmied, H., Stacke, T., Telteu, C.-E., Thiery, W., Veldkamp, T., Zhao, F., and Wada, Y.: Global terrestrial water storage and drought severity under climate change, Nature Climate Change, 1–8, https://doi.org/10.1038/s41558-020-00972-w, 2021.
Satoh, Y., Yoshimura, K., Pokhrel, Y., Kim, H., Shiogama, H., Yokohata, T., Hanasaki, N., Wada, Y., Burek, P., Byers, E., Schmied, H. M., Gerten, D., Ostberg, S., Gosling, S. N., Boulange, J. E. S., and Oki, T.: The timing of unprecedented hydrological drought under climate change, Nat Commun, 13, 3287, https://doi.org/10.1038/s41467-022-30729-2, 2022.
Shrestha, R. R., Cannon, A. J., Schnorbus, M. A., and Alford, H.: Climatic Controls on Future Hydrologic Changes in a Subarctic River Basin in Canada, J. Hydrometeor., 20, 1757–1778, https://doi.org/10.1175/JHM-D-18-0262.1, 2019.
Telteu, C.-E., Müller Schmied, H., Thiery, W., Leng, G., Burek, P., Liu, X., Boulange, J. E. S., Andersen, L. S., Grillakis, M., Gosling, S. N., Satoh, Y., Rakovec, O., Stacke, T., Chang, J., Wanders, N., Shah, H. L., Trautmann, T., Mao, G., Hanasaki, N., Koutroulis, A., Pokhrel, Y., Samaniego, L., Wada, Y., Mishra, V., Liu, J., Döll, P., Zhao, F., Gädeke, A., Rabin, S. S., and Herz, F.: Understanding each other’s models: an introduction and a standard representation of 16 global water models to support intercomparison, improvement, and communication, Geoscientific Model Development, 14, 3843–3878, https://doi.org/10.5194/gmd-14-3843-2021, 2021.
Citation: https://doi.org/10.5194/egusphere-2024-3170-AC2
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-3170', Anonymous Referee #1, 06 Jan 2025
The study by Shrestha et al. presents a benchmarking analysis of a global hydrological model (CWATM) against a watershed hydrological model (VIC) for future projections. The paper is well-structured, clearly written, and achieves its stated goals effectively. The comparative approach appears robust and provides valuable insights into the models' performances and I especially appreciate that uncertainties are being clearly discussed.
The study would benefit from even more emphasising how the models' water balance components compare to real-world observations. In Section 4.4, adding comparisons to observed data for key water balance components such as annual (monthly) streamflow and the distribution of precipitation into evaporation and runoff, would help contextualize each models’ performance against real-world observations and each other.
Additionally, while the abstract highlights the identification of areas for model improvement (l. 22-23) and the introduction points towards limitations of these models, the potential areas for improvement are not as clearly outlined in the conclusion. To do this, however, could enhance the study's findings and clarity.Technical comments
- Improve the placement of Tables 1, 2, and 3 and Figure 6 to enhance clarity and avoid confusion. The items are not exactly located where they are mentioned in the text, which I found confusing at times.
- Clarify in Table 1 that Manning’s n refers to a roughness coefficient.
- Provide a complete caption for Figure 6 to ensure it can stand independently, even if it is similar to Figure 5.
- Consider providing consistent ‘units’ in Table 1, since the table features arc-minutes, arc-seconds and degrees for grid-scale size references. This makes it difficult to get a good idea of the actual differences in scale.
Citation: https://doi.org/10.5194/egusphere-2024-3170-RC1 -
AC1: 'Reply on RC1', Rajesh Shrestha, 26 Feb 2025
Response to Anonymous Referee #1 (shown in bold and italic)
The study by Shrestha et al. presents a benchmarking analysis of a global hydrological model (CWATM) against a watershed hydrological model (VIC) for future projections. The paper is well-structured, clearly written, and achieves its stated goals effectively. The comparative approach appears robust and provides valuable insights into the models' performances and I especially appreciate that uncertainties are being clearly discussed.
We thank the reviewer for taking time to review our manuscript and positive comments. We will carefully address all comments in the revised manuscript. At this stage, we would like to focus our response on the main points raised by the reviewer.
The study would benefit from even more emphasising how the models' water balance components compare to real-world observations. In Section 4.4, adding comparisons to observed data for key water balance components such as annual (monthly) streamflow and the distribution of precipitation into evaporation and runoff, would help contextualize each models’ performance against real-world observations and each other.
We agree with the reviewer that comparing historical and projected changes with observations will be beneficial to contextualize the results against real world observations. To that end, we will add observed streamflow in Section 4.4 Figure, and discuss the models’ performance against observations. However, evaporation observations are not available to compare with the model results.
Additionally, while the abstract highlights the identification of areas for model improvement (l. 22-23) and the introduction points towards limitations of these models, the potential areas for improvement are not as clearly outlined in the conclusion. To do this, however, could enhance the study's findings and clarity.
We also agree with the reviewer that it will be beneficial to outline potential areas of model improvements. Based on the study findings, we will provide an outline of potential improvements in both CWatM and VIC models in the conclusions section.
Technical comments
- Improve the placement of Tables 1, 2, and 3 and Figure 6 to enhance clarity and avoid confusion. The items are not exactly located where they are mentioned in the text, which I found confusing at times.
- Clarify in Table 1 that Manning’s n refers to a roughness coefficient.
- Provide a complete caption for Figure 6 to ensure it can stand independently, even if it is similar to Figure 5.
- Consider providing consistent ‘units’ in Table 1, since the table features arc-minutes, arc-seconds and degrees for grid-scale size references. This makes it difficult to get a good idea of the actual differences in scale.
We will make all suggested technical changes in the revised manuscript.
Citation: https://doi.org/10.5194/egusphere-2024-3170-AC1
-
RC2: 'Comment on egusphere-2024-3170', Anonymous Referee #2, 23 Jan 2025
I enjoyed reviewing the manuscript "Benchmarking historical performance and future projections from a global hydrologic model with a basin-scale model" by Rajesh R. Shrestha and co-authors and I hope that my comments will help to further improve it. In this study, the authors compare simulations and predictions of two hydrological models (the Community Water Model, CWatM, and Variable Infiltration Capacity, VIC), both calibrated for the Canadian Liard River basin. The authors found satisfactory model performances of both models when simulating a historical time period. When using the models to predict changes under different future scenarios, the results of both models were mainly consistent. The manuscript is understandable and interesting, as well as clearly and concisely written.
General comments
In general, I think a clarification on the definition of a global hydrologic model (GHM) is required. In the manuscript, the CWatM is considered as a global model that is compared to the VIC, a watershed model (WHM). However, according to the model development paper (Burek et al., 2020), the CWatM is not solely a global, but also a regional model. This is also stated in Sect. 3.1, lines 109-110. Since here, the CWatM is calibrated to one specific basin, one could argue that it is used as a regional model and not as a global model. In fact, the main difference between the application of the CWatM and the VIC seems to be the number of subcatchments for which the models were calibrated. This is still a valuable comparison, but needs to be framed differently (i.e., not as a comparison of a GHM to a WHM) in my opinion. I recommend to either explain more clearly why the CWatM can still be considered a GHM in this application case, or to adapt the goal of the manuscript accordingly. Related to this, it may be interesting to also include how well a model performs (or, how well the two models perform) for this basin when not specifically calibrated as a comparison, i.e., as a counterpart to the benchmark of the calibrated WHM.
Furthermore, I see a general challenge in how the calibration process was designed: According to Table 1, the calibration of the VIC takes the NSE, LNSE, and VB into account, while the CWatM is calibrated on the KGE. In my opinion, both models should be treated analogously during the calibration process, i.e., calibrated on the same objective function. Currently, the different criteria during calibration lead to additional uncertainties regarding the differences in model performance that can be avoided when the same criteria are used. Except if there is a reason for the different treatment of the two models (that should then be stated clearly, and the effects on the results should be included in the discussion), one of the model calibrations needs to be repeated with the objective function that was used for the other model. All analyses should then be based on those results.
Specific comments
- In lines 115-116, consider including the paper by Hanus et al. (2024, 10.5194/gmd-17-5123-2024) on coupling the CWatM with a glacier model as an example how the CWatM should become more feasible for the application in cold regions.
- Between Sect. 3.2 and Sect. 3.3, I am missing some information on how the models were calibrated. This information is available in Table 1, but it would be helpful to at least have a remark on where to find this information between the explanation on the forcing data and the evaluation methods.
- When providing the reader with the model performance results (currently Table 3), it will be important to indicate which of the performance measures were used in the calibration process, i.e., which of the measures were optimized (see general comment on the different objective functions).
- Please include the black square in the legend in Figure 3. Please also add in the caption of Figure 3 what the different datapoints per column, colour, and symbol (e.g., the different purple triangles in the same column) refer to. Furthermore, in Figure 3 consider to: not capitalize “temperature” and “precipitation”, add a space before “°C” in the legend, and write the name of the SSPs consistently in the text (lines 155 and 196), in the caption and in the legend.
- Please include in the caption of Figure 5 that you did not include the summer months on purpose (I assume because they are not interesting regarding the SWE).
- Currently, Sect. 4 is entitled “Results and discussion” and Sect. 5 is entitled “Discussion and Conclusions”. Please make sure that you clearly indicate where your results are discussed. I suggest you to follow the common standard of first presenting your results (Sect. 4) and then discussing them (Sect. 5), i.e., to move all discussion parts currently included in Sect. 4 to Sect. 5 and thus have only one comprehensive discussion. Concluding remarks could be added to Sect. 5 (“Discussion and conclusions”) or as a separate Sect. 6.
- The study is based on one basin only. This has the advantage that detailed analyses are possible, and different processes (as opposed to only model performances regarding streamflow simulation) can be considered. This is nicely done in the manuscript. However, thanks to the growing computational power, model comparison studies can nowadays rather easily be based on a larger number of catchments. Such studies have the advantage that the results are more robust and can be generalized, as the variability between different catchments can be obtained. In a study with only one catchment, this is not possible. Please add this in the discussion, i.e., make sure to emphasize that it is likely possible that the results are different in other application cases.
Technical suggestions
- Consider including axis labels, a north arrow, and a scale bar in Figure 1.
- For consistency, in the title of Sect. 3 “Models, Data and Analyses”, the terms “data” and “analyses” should not be capitalized.
- I agree with the first reviewer that a common unit for the different entries in Table 1 would make comparison easier. I suggest to add this as a complement to the units that are given in this table now (i.e., also keep the units that are given with each dataset).
- In the column “spatial resolution”, for VIC, the word “for” before “snow” seems to be missing.
- In Sect. 1 and Sect. 4.1, for example, you use “x” as a sign for multiplication, in Sect. 4.5 you use “*”. Consider changing this for consistency.
- For all occurrences of the colour-bar legends (median change and model agreement), consider replacing the term “model agreement” with “GCM agreement” to avoid confusion with the agreement of the two models VIC and CWatM.
- The Oxford comma is not used consistently, this could be improved.
- Consider including the full term in addition to the abbreviation when using an abbreviation in a Figure or Table caption (as you did it for example for “GWL” in the caption of Figure 4).
- Considering the facts that there are already many abbreviations in the manuscript and that you do not make use of the abbreviation “LRB” very often, I recommend that you do not use an abbreviation for the Liard River basin at all. Similarly, for “GEV” abbreviating “generalized extreme value”: After introducing this abbreviation, it is not used anymore, so I would leave it away.
- You sometimes use “()” around units in Figures and sometimes “[]”, this may be improved for consistency. Similarly, you should use “s” instead of “sec” as an abbreviation of “second” and potentially also decide for either “discharge” or “flow” in the different plots showing discharge or flow values.
Citation: https://doi.org/10.5194/egusphere-2024-3170-RC2 -
AC2: 'Reply on RC2', Rajesh Shrestha, 26 Feb 2025
Response to Anonymous Referee #2 (shown in bold and italic)
I enjoyed reviewing the manuscript "Benchmarking historical performance and future projections from a global hydrologic model with a basin-scale model" by Rajesh R. Shrestha and co-authors and I hope that my comments will help to further improve it. In this study, the authors compare simulations and predictions of two hydrological models (the Community Water Model, CWatM, and Variable Infiltration Capacity, VIC), both calibrated for the Canadian Liard River basin. The authors found satisfactory model performances of both models when simulating a historical time period. When using the models to predict changes under different future scenarios, the results of both models were mainly consistent. The manuscript is understandable and interesting, as well as clearly and concisely written.
We thank the reviewer for taking time to review our manuscript and constructive comments. We will carefully address all comments in the revised manuscript. At this stage, we would like to focus our response on the main points raised by the reviewer.
General comments
In general, I think a clarification on the definition of a global hydrologic model (GHM) is required. In the manuscript, the CWatM is considered as a global model that is compared to the VIC, a watershed model (WHM). However, according to the model development paper (Burek et al., 2020), the CWatM is not solely a global, but also a regional model. This is also stated in Sect. 3.1, lines 109-110. Since here, the CWatM is calibrated to one specific basin, one could argue that it is used as a regional model and not as a global model. In fact, the main difference between the application of the CWatM and the VIC seems to be the number of subcatchments for which the models were calibrated. This is still a valuable comparison, but needs to be framed differently (i.e., not as a comparison of a GHM to a WHM) in my opinion. I recommend to either explain more clearly why the CWatM can still be considered a GHM in this application case, or to adapt the goal of the manuscript accordingly.
Based on the reviewer’s comment, we feel that it is important to distinguish the model application from the model itself when describing the global hydrologic model (GHM). As the reviewer pointed out, CWatM is designed for global and regional assessments (Burek et al., 2020), however, it not tailored to a specific region or a basin. Furthermore, it can simulate terrestrial water cycle processes and human usage on a global scale due to the incorporation of global scale datasets (Burek et al., 2020; Telteu et al., 2021). CWatM has also been used in several global scale assessments as a part of a multi-GHM ensemble (e.g. Boulange et al., 2021; Pokhrel et al., 2021; Satoh et al., 2022). Due to these reasons, we feel that CWatM should classified as a GHM. However, we agree with the reviewer that our application is actually a basin scale. In fact, the main objective of our study is to assess the capability of a GHM – not tailored to represent basin scale processes (specifically, cold-regions processes) - in replicating historical simulations and future projections from a watershed hydrologic model. In the revised version, we will reframe our study by adding these clarifications.
Related to this, it may be interesting to also include how well a model performs (or, how well the two models perform) for this basin when not specifically calibrated as a comparison, i.e., as a counterpart to the benchmark of the calibrated WHM.
The reviewer made an interesting suggestion to compare uncalibrated models as an additional benchmarking exercise. However, the performance of an uncalibrated model will depend on the set of uncalibrated parameters used, as the model with one set of uncalibrated parameters could produce totally different results compared to the model with another set of uncalibrated parameters. Due to this reason, we do not think comparing calibrated and uncalibrated model results will provide a fair assessment of the model’s capability.
Furthermore, I see a general challenge in how the calibration process was designed: According to Table 1, the calibration of the VIC takes the NSE, LNSE, and VB into account, while the CWatM is calibrated on the KGE. In my opinion, both models should be treated analogously during the calibration process, i.e., calibrated on the same objective function. Currently, the different criteria during calibration lead to additional uncertainties regarding the differences in model performance that can be avoided when the same criteria are used. Except if there is a reason for the different treatment of the two models (that should then be stated clearly, and the effects on the results should be included in the discussion), one of the model calibrations needs to be repeated with the objective function that was used for the other model. All analyses should then be based on those results.
With regards to the reviewer’s suggestion of using the same objective function for an analogous calibration of both models, we would like to clarify that we used the calibrated VIC model with three objective functions (NSE, LNSE and VB) from our previous study (Shrestha et al., 2019), while CWatM was calibrated by using the KGE objective function (which is a combination of correlation, bias ratio and variability ratio) as designed for a typical calibration setup by the model developers (Burek et al., 2020). While we agree that the use of different objective functions could add some uncertainty, we would like to note that the selection of an objective function is only a part of the challenge for model calibration. More importantly, several different parameter combinations within a chosen model structure could yield similar performance due to model equifinality (Beven, 2006). Given this unavoidable challenge, we took additional steps for selecting the “best parameter sets” from the calibration runs. Specifically, we ran the NSGA-II calibration setups for both models at least three times and considered the models’ ability to simulate peak flow, low flow and water balance by visualizing the hydrographs and annual water balance obtained from top parameter sets from each calibration run. We used the modeller’s judgement for selecting the “best parameter set”, which is not necessarily the parameter set with the best score for one of the objective functions considered. The effect of this additional step can be seen in the performance the two models for the Liard-UC station over the calibration period. Specifically, although the CWatM model for this station was calibrated by using the KGE objective function, the KGE performance for the selected CWatM model parameters is inferior to that of VIC (KGE was not used for VIC calibration), while the NSE and LNSE (used in calibration of VIC but not CWatM) performances of CWatM are similar to VIC. Thus, our calibration can be described as automatic calibration supplemented by the modeller’s judgement. We did not provide these details in the original version of manuscript, with a thinking that model calibration is not the main focus of our study. However, the reviewer has correctly pointed out the need to provide further details on calibration. In the revised version, we will add a paragraph detailing the points outlined above.
Specific comments
- In lines 115-116, consider including the paper by Hanus et al. (2024, 10.5194/gmd-17-5123-2024) on coupling the CWatM with a glacier model as an example how the CWatM should become more feasible for the application in cold regions.
Thank you for the suggestion, we will reference this study as an application of the CWatM model for cold regions.
- Between Sect. 3.2 and Sect. 3.3, I am missing some information on how the models were calibrated. This information is available in Table 1, but it would be helpful to at least have a remark on where to find this information between the explanation on the forcing data and the evaluation methods.
As discussed above, we will add a paragraph outlining our calibration steps.
- When providing the reader with the model performance results (currently Table 3), it will be important to indicate which of the performance measures were used in the calibration process, i.e., which of the measures were optimized (see general comment on the different objective functions).
We agree with the reviewer, and we will indicate the performance metrics used as objective functions in the calibration of each model in Table 3.
- Please include the black square in the legend in Figure 3. Please also add in the caption of Figure 3 what the different datapoints per column, colour, and symbol (e.g., the different purple triangles in the same column) refer to. Furthermore, in Figure 3 consider to: not capitalize “temperature” and “precipitation”, add a space before “°C” in the legend, and write the name of the SSPs consistently in the text (lines 155 and 196), in the caption and in the legend.
We agree with the reviewer, and will make the suggested technical edits in Figure 3 legend and caption.
- Please include in the caption of Figure 5 that you did not include the summer months on purpose (I assume because they are not interesting regarding the SWE).
We will add an explanation in the Figure 5 caption that summer months were not included in the Figure because of the lack of snow.
- Currently, Sect. 4 is entitled “Results and discussion” and Sect. 5 is entitled “Discussion and Conclusions”. Please make sure that you clearly indicate where your results are discussed. I suggest you to follow the common standard of first presenting your results (Sect. 4) and then discussing them (Sect. 5), i.e., to move all discussion parts currently included in Sect. 4 to Sect. 5 and thus have only one comprehensive discussion. Concluding remarks could be added to Sect. 5 (“Discussion and conclusions”) or as a separate Sect. 6.
It was our intention our to provide “Results and Discussion” together in section 4, and “Summary and Conclusions” in section 5, and we structured the sections accordingly. It was an oversight on our part that section 5 was mistakenly entitled “Discussion and Conclusions”. We will use the correct heading for section 5 in the revised version of the manuscript.
- The study is based on one basin only. This has the advantage that detailed analyses are possible, and different processes (as opposed to only model performances regarding streamflow simulation) can be considered. This is nicely done in the manuscript. However, thanks to the growing computational power, model comparison studies can nowadays rather easily be based on a larger number of catchments. Such studies have the advantage that the results are more robust and can be generalized, as the variability between different catchments can be obtained. In a study with only one catchment, this is not possible. Please add this in the discussion, i.e., make sure to emphasize that it is likely possible that the results are different in other application cases.
The reviewer raised an important point regarding generalization of the results, given our study is focused on a single basin. Although our study basin Liard is large (about 275,000 km2), and consistent results for Liard-UC subbasin improve confidence in our results, we agree that broader generalization may not possible. We will discuss this point in the conclusions section.
Technical suggestions
- Consider including axis labels, a north arrow, and a scale bar in Figure 1.
- For consistency, in the title of Sect. 3 “Models, Data and Analyses”, the terms “data” and “analyses” should not be capitalized.
- I agree with the first reviewer that a common unit for the different entries in Table 1 would make comparison easier. I suggest to add this as a complement to the units that are given in this table now (i.e., also keep the units that are given with each dataset).
- In the column “spatial resolution”, for VIC, the word “for” before “snow” seems to be missing.
- In Sect. 1 and Sect. 4.1, for example, you use “x” as a sign for multiplication, in Sect. 4.5 you use “*”. Consider changing this for consistency.
- For all occurrences of the colour-bar legends (median change and model agreement), consider replacing the term “model agreement” with “GCM agreement” to avoid confusion with the agreement of the two models VIC and CWatM.
- The Oxford comma is not used consistently, this could be improved.
- Consider including the full term in addition to the abbreviation when using an abbreviation in a Figure or Table caption (as you did it for example for “GWL” in the caption of Figure 4).
- Considering the facts that there are already many abbreviations in the manuscript and that you do not make use of the abbreviation “LRB” very often, I recommend that you do not use an abbreviation for the Liard River basin at all. Similarly, for “GEV” abbreviating “generalized extreme value”: After introducing this abbreviation, it is not used anymore, so I would leave it away.
- You sometimes use “()” around units in Figures and sometimes “[]”, this may be improved for consistency. Similarly, you should use “s” instead of “sec” as an abbreviation of “second” and potentially also decide for either “discharge” or “flow” in the different plots showing discharge or flow values.
We will address all technical suggestions in the revised manuscript.
References
Beven, K.: A manifesto for the equifinality thesis, Journal of Hydrology, 320, 18–36, https://doi.org/10.1016/j.jhydrol.2005.07.007, 2006.
Boulange, J., Hanasaki, N., Satoh, Y., Yokohata, T., Shiogama, H., Burek, P., Thiery, W., Gerten, D., Schmied, H. M., Wada, Y., Gosling, S. N., Pokhrel, Y., and Wanders, N.: Validity of estimating flood and drought characteristics under equilibrium climates from transient simulations, Environ. Res. Lett., 16, 104028, https://doi.org/10.1088/1748-9326/ac27cc, 2021.
Burek, P., Satoh, Y., Kahil, T., Tang, T., Greve, P., Smilovic, M., Guillaumot, L., Zhao, F., and Wada, Y.: Development of the Community Water Model (CWatM v1.04) – a high-resolution hydrological model for global and regional assessment of integrated water resources management, Geoscientific Model Development, 13, 3267–3298, https://doi.org/10.5194/gmd-13-3267-2020, 2020.
Pokhrel, Y., Felfelani, F., Satoh, Y., Boulange, J., Burek, P., Gädeke, A., Gerten, D., Gosling, S. N., Grillakis, M., Gudmundsson, L., Hanasaki, N., Kim, H., Koutroulis, A., Liu, J., Papadimitriou, L., Schewe, J., Müller Schmied, H., Stacke, T., Telteu, C.-E., Thiery, W., Veldkamp, T., Zhao, F., and Wada, Y.: Global terrestrial water storage and drought severity under climate change, Nature Climate Change, 1–8, https://doi.org/10.1038/s41558-020-00972-w, 2021.
Satoh, Y., Yoshimura, K., Pokhrel, Y., Kim, H., Shiogama, H., Yokohata, T., Hanasaki, N., Wada, Y., Burek, P., Byers, E., Schmied, H. M., Gerten, D., Ostberg, S., Gosling, S. N., Boulange, J. E. S., and Oki, T.: The timing of unprecedented hydrological drought under climate change, Nat Commun, 13, 3287, https://doi.org/10.1038/s41467-022-30729-2, 2022.
Shrestha, R. R., Cannon, A. J., Schnorbus, M. A., and Alford, H.: Climatic Controls on Future Hydrologic Changes in a Subarctic River Basin in Canada, J. Hydrometeor., 20, 1757–1778, https://doi.org/10.1175/JHM-D-18-0262.1, 2019.
Telteu, C.-E., Müller Schmied, H., Thiery, W., Leng, G., Burek, P., Liu, X., Boulange, J. E. S., Andersen, L. S., Grillakis, M., Gosling, S. N., Satoh, Y., Rakovec, O., Stacke, T., Chang, J., Wanders, N., Shah, H. L., Trautmann, T., Mao, G., Hanasaki, N., Koutroulis, A., Pokhrel, Y., Samaniego, L., Wada, Y., Mishra, V., Liu, J., Döll, P., Zhao, F., Gädeke, A., Rabin, S. S., and Herz, F.: Understanding each other’s models: an introduction and a standard representation of 16 global water models to support intercomparison, improvement, and communication, Geoscientific Model Development, 14, 3843–3878, https://doi.org/10.5194/gmd-14-3843-2021, 2021.
Citation: https://doi.org/10.5194/egusphere-2024-3170-AC2
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 350 | 73 | 20 | 443 | 43 | 18 | 29 |
- HTML: 350
- PDF: 73
- XML: 20
- Total: 443
- Supplement: 43
- BibTeX: 18
- EndNote: 29
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Rajesh R. Shrestha
Alex J. Cannon
Sydney Hoffman
Marie Whibley
Aranildo Lima
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2325 KB) - Metadata XML
-
Supplement
(671 KB) - BibTeX
- EndNote
- Final revised paper