the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 03 May 2023
Evaluation of hydrological models on small mountainous catchments: impact of the meteorological forcings
Abstract. Hydrological modelling of small mountainous catchments is particularly challenging because of the high spatio-temporal resolution required for the meteorological forcings. In-situ measurements of precipitation are typically scarce in these remote areas, particularly at high elevations. Precipitation reanalyses propose different alternative forcings for the simulation of streamflow using hydrological models. In this paper, we evaluate the performances of two hydrological models representing some of the key processes for small mountainous catchments, using different meteorological products with a fine spatial and temporal resolution. The evaluation is performed on 55 small catchments of the Northern French Alps. While the simulated streamflows are adequately reproduced for most of the configurations, these evaluations emphasize the added value of radar measurements, in particular for the reproduction of flood events. However, these better performances are only obtained because the hydrological models correct the underestimations of accumulated amounts (e.g. annual) from the radar data in high-elevation areas.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(2701 KB)
-
Supplement
(10151 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2701 KB) - Metadata XML
-
Supplement
(10151 KB) - BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
CC1: 'Comment on egusphere-2023-845', Guillaume Thirel, 16 May 2023
Dear authors, dear editor.
I would like to make a very small and specific comment about a few lines of this manuscript (please note I belong to the same institutions than some of the authors, but I think that this input might be useful to avoid any misunderstanding regarding what the authors wrote).
Indeed, at lines 226-231, the authors state: "Several studies have discussed the pros and cons of two popular criteria: The Nash-Sutcliffe efficiency (NSE Nash and Sutcliffe, 1970) and the Kling-Gupta Efficiency (KGE Gupta et al., 2009). Recently, Clark et al. (2021) emphasize the fact that these criteria rely on squared errors between simulated and observed streamflows and are subject to considerable sampling uncertainties. Large differences between observations and simulations are amplified by these squared errors and different remedies have been proposed to reduce their influence on the calibration process (e.g., log transformation, see Santos et al., 2018)."
I believe the formulation of these sentences should be improved. Indeed, in Santos et al. (2018), we explained and showed why the log-transformation should NOT be used with the KGE. Meanwhile, what the authors wrote could be read as we suggested to use the log transformation for the NSE and KGE criteria. I would just like to avoid this miunderstanding.
RegardsCitation: https://doi.org/10.5194/egusphere-2023-845-CC1 -
AC2: 'Reply on CC1', Guillaume Evin, 04 Sep 2023
Dear colleague,
Thank you for this comment. We fully agree that the sentence was misleading and it was removed from the manuscript as this paragraph concerned only the criteria used for evaluation (and not calibration). Therefore, the technical discussion about the transformation that should or should not be applied to the KGE for calibration was not really necessary here.
Citation: https://doi.org/10.5194/egusphere-2023-845-AC2
-
AC2: 'Reply on CC1', Guillaume Evin, 04 Sep 2023
-
RC1: 'Comment on egusphere-2023-845', Anonymous Referee #1, 30 Jun 2023
Review for “Evaluation of hydrological models on small mountainous catchments: impact of the meteorological forcings” by Evil et al.
I apologize with the authors for the delay in sending my revision. Here my comments to your work.
The paper analyses the use of four different meteorological reanalysis products, with high spatial and temporal resolution, for testing the performances of two hydrological models. The study considers 55 catchments of different size and elevation, mostly located in a mountainous region in France. The evaluation, based on both streamflow and flood events, show better performances for the product including both radar and gauges. This show the added value of considering radar. The authors also highlight how this product underestimates precipitation at high elevations, and the good performance in the hydrological model depend on a correction parameter used by the model.
Based on my personal reading, the presented analysis is of interest for hydrological modelling of ungauged catchments. I believe that the manuscript is well written and organized, and it deserves to be published in NHESS, after minor revisions listed below.
- Line 56: “less than 200 km2” … you mention 300km2 at line 69
- Line 66. I suggest to add (in this section 2.1 or at least in the supplemental) a table with the list of the catchments with their main features (numbering, identification name, area, elevation range or average value, …). If possible, consider to report the number of each catchment in the map in figure 1, and in the explanations of results and discussion together with the catchment name. This could help the reader in identify the catchments and their main features when reading the following results and discussion section. I was sometimes lost with the different names of the catchments.
- Line 225. In all the equations in section 4.1, are you using absolute differences?
- Line 251-258. You define some errors (let’s say E, where E = PFE, TPE, VE) using absolute differences, then you transform them as 1- E. I have to comments here: i) why not just considering E with sign, for having a measure of the direction of the error (under- or overestimation)?; ii) I found confusing the use of both E and 1-E in the explanations of the results (for example, lines 279-289). I suggest to use just one.
- Line 270-275. i) For better compare the results, I suggest to use same y-axis limits for the plots referring to the same type of index. For example, for mNSE in panels a-d, for QRE in panels e-h, … ii) maybe a comment is needed about SMASH model in panel d), showing huge range of mNSE compared to MORDOR-SD
- Line 439: You mention some problems in high-elevation catchments, and (line 468) that COMESPHORE underestimates precipitation in high elevation areas. Consider to add a plot of mNSE vs elevation, for example as panels in figure4, to synthesis the maps and made more evident (if there is) a relation of mNSE with elevation.
- Line 457: “small mountainous” maybe is “small catchments”.
- Supplemental, Figure S36-37. Same color scales in the panels could help in the comparisons.
- Supplemental, line 16. “than with COMEPHORE” is maybe “MORDOR-SD”
Citation: https://doi.org/10.5194/egusphere-2023-845-RC1 -
AC1: 'Reply on RC1', Guillaume Evin, 04 Sep 2023
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-845/egusphere-2023-845-AC1-supplement.pdf
-
RC2: 'Comment on egusphere-2023-845', Anonymous Referee #2, 04 Jul 2023
This very well written paper addresses the important question of how to model streamflow in small catchments with different precipitation products. Overall, the paper is a bit limited in terms of references to existing literature on the question of how input resolution interacts with model performance. Also, it does not discuss what different strategies actually exist to infer the hydro model meteorology at the appropriate resolution, from a meterological product that has a different resolution and a different model topography. It appears to me that an essential modelling choice is missing: how to combine the meteo product with the model? There is one option presented per hydro model but we do not know if this is a heuristic choice or the best option or what the literatur says about this. In general, I do e.g. not think that retaining simply the meteo pixels within a catchment is the best option (but perhaps it is for rainfall?). There is a short discussion on the absence of a precip gradient for one product but perhaps it would be good to have a more systematic discussion of how to create the hydro model meteo based on the input meteo.
Also, I strongly recommend to make much clearer what take-aways are relevant beyond the studied catchments and how new they are (at the moment, there are two take-aways which do not reveal completely new insights). And: what does all this for the modelling of ungauged catchments, for which no correction parameters can be calibrated? Would be cool to have some input on this question.
Detailed comments:
- From the abstract, the actual innovation is not clear, it seems like another paper on a previously often studied topic; what is small?
- Very few references at the start of the intro, little reference to extensive literature on the role of spatial resolution of input (rainfall) forcing on the quality of hydrological simulations
- From the methods, I understand that the spatial input product is aggregated to the hydro model by taking the meteo values per pixel: is this a good strategy given that the meteo product does not represent the true topography of the catchment ? In particular for the model that uses elevation bands? How are the pixel of the meteo product matched with the pixels of the hydro model for the other model?
- From the methods it seems like the two models are not calibrated with the same criterion, why? Is this a good idea? does this impact the analysis beyond what is mentioned in the paper?
- The paper would benefit from a concise summary of how the two models differ in terms of process representation? How do they estimate evaporation? Evaporation is almost not mentioned in the entire paper? But it should have a key influence on the representation of the water balance?
- Section 3: how do the model topographies of the meteo products differ?
- Section 4: do I understand correctly that the models are calibrated with criteria that are discussed in the sections dedicated to the models and that other criteria are used to evaluate the performance? Or are the criteria in the model section not relevant? Are the models calibrated with each meteo product? And why is the error on the floods not simply computed as a square-error, is NSE appropriate for this kind of signal? Are the values comparable to those of an entire year?
- Results: are the mNSE values a priori comparable for the different catchments? Since we do not see any streamflow time series and do not know if there are differences in the regimes, it is hard to judge
- please also see a few comments in the annotated pdf
-
AC3: 'Reply on RC2', Guillaume Evin, 04 Sep 2023
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-845/egusphere-2023-845-AC3-supplement.pdf
-
RC3: 'Comment on egusphere-2023-845', Anonymous Referee #3, 23 Jul 2023
The manuscript by Evin et al., assess the performance of different precipitation products in 55 mountainous basins in France, with a specific focus on flood. They conclude that radar measurements are helpful to capture finer scale events leading to flooding, but mountain precipitation is not well captured with radar. This is a clear and straightforward case study that provide incremental insights on precipitation products in mountainous basin. I have a few minor comments below:
I am a bit concerned with the scope of the objective to “choose the best product”. This is very limiting as an objective as it has very limited application for a wider audience. I would reword this objective to something more applicable to a wider range of studies. I suggest shifting the focus of the paper to be about the value added of radar information in mountain basins, as showcased by the analysis of the 55 basins, instead of having a primary objective to “select the best product”.
In the introduction, I would like to see a more robust presentation of radar measurements for precipitation in mountain regions, specifically to how it performs with snow measurements.
I found the most interesting part of the paper to be the discussion. Specifically, the analysis of model performance with different products is linked to process representation in the model (groundwater loss) and precipitation and elevation representation. I would like to see some more information on radar performance for snow vs. rain at higher elevations, and if that could cause some of these issues. Fig 8b also suggests that ERA is actually quite good at capturing high-elevation precipitation, which is a strength that could be mentioned in the conclusion. The groundwater section could also be clearer: Do you mean water is exiting the basin as groundwater, so you have to reduce precipitation? It would be interesting to have more information on how this lack of process representation could be fixed, and what would be advantages of using a model with groundwater processes included in the study.
Citation: https://doi.org/10.5194/egusphere-2023-845-RC3 -
AC4: 'Reply on RC3', Guillaume Evin, 04 Sep 2023
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-845/egusphere-2023-845-AC4-supplement.pdf
-
AC4: 'Reply on RC3', Guillaume Evin, 04 Sep 2023
Interactive discussion
Status: closed
-
CC1: 'Comment on egusphere-2023-845', Guillaume Thirel, 16 May 2023
Dear authors, dear editor.
I would like to make a very small and specific comment about a few lines of this manuscript (please note I belong to the same institutions than some of the authors, but I think that this input might be useful to avoid any misunderstanding regarding what the authors wrote).
Indeed, at lines 226-231, the authors state: "Several studies have discussed the pros and cons of two popular criteria: The Nash-Sutcliffe efficiency (NSE Nash and Sutcliffe, 1970) and the Kling-Gupta Efficiency (KGE Gupta et al., 2009). Recently, Clark et al. (2021) emphasize the fact that these criteria rely on squared errors between simulated and observed streamflows and are subject to considerable sampling uncertainties. Large differences between observations and simulations are amplified by these squared errors and different remedies have been proposed to reduce their influence on the calibration process (e.g., log transformation, see Santos et al., 2018)."
I believe the formulation of these sentences should be improved. Indeed, in Santos et al. (2018), we explained and showed why the log-transformation should NOT be used with the KGE. Meanwhile, what the authors wrote could be read as we suggested to use the log transformation for the NSE and KGE criteria. I would just like to avoid this miunderstanding.
RegardsCitation: https://doi.org/10.5194/egusphere-2023-845-CC1 -
AC2: 'Reply on CC1', Guillaume Evin, 04 Sep 2023
Dear colleague,
Thank you for this comment. We fully agree that the sentence was misleading and it was removed from the manuscript as this paragraph concerned only the criteria used for evaluation (and not calibration). Therefore, the technical discussion about the transformation that should or should not be applied to the KGE for calibration was not really necessary here.
Citation: https://doi.org/10.5194/egusphere-2023-845-AC2
-
AC2: 'Reply on CC1', Guillaume Evin, 04 Sep 2023
-
RC1: 'Comment on egusphere-2023-845', Anonymous Referee #1, 30 Jun 2023
Review for “Evaluation of hydrological models on small mountainous catchments: impact of the meteorological forcings” by Evil et al.
I apologize with the authors for the delay in sending my revision. Here my comments to your work.
The paper analyses the use of four different meteorological reanalysis products, with high spatial and temporal resolution, for testing the performances of two hydrological models. The study considers 55 catchments of different size and elevation, mostly located in a mountainous region in France. The evaluation, based on both streamflow and flood events, show better performances for the product including both radar and gauges. This show the added value of considering radar. The authors also highlight how this product underestimates precipitation at high elevations, and the good performance in the hydrological model depend on a correction parameter used by the model.
Based on my personal reading, the presented analysis is of interest for hydrological modelling of ungauged catchments. I believe that the manuscript is well written and organized, and it deserves to be published in NHESS, after minor revisions listed below.
- Line 56: “less than 200 km2” … you mention 300km2 at line 69
- Line 66. I suggest to add (in this section 2.1 or at least in the supplemental) a table with the list of the catchments with their main features (numbering, identification name, area, elevation range or average value, …). If possible, consider to report the number of each catchment in the map in figure 1, and in the explanations of results and discussion together with the catchment name. This could help the reader in identify the catchments and their main features when reading the following results and discussion section. I was sometimes lost with the different names of the catchments.
- Line 225. In all the equations in section 4.1, are you using absolute differences?
- Line 251-258. You define some errors (let’s say E, where E = PFE, TPE, VE) using absolute differences, then you transform them as 1- E. I have to comments here: i) why not just considering E with sign, for having a measure of the direction of the error (under- or overestimation)?; ii) I found confusing the use of both E and 1-E in the explanations of the results (for example, lines 279-289). I suggest to use just one.
- Line 270-275. i) For better compare the results, I suggest to use same y-axis limits for the plots referring to the same type of index. For example, for mNSE in panels a-d, for QRE in panels e-h, … ii) maybe a comment is needed about SMASH model in panel d), showing huge range of mNSE compared to MORDOR-SD
- Line 439: You mention some problems in high-elevation catchments, and (line 468) that COMESPHORE underestimates precipitation in high elevation areas. Consider to add a plot of mNSE vs elevation, for example as panels in figure4, to synthesis the maps and made more evident (if there is) a relation of mNSE with elevation.
- Line 457: “small mountainous” maybe is “small catchments”.
- Supplemental, Figure S36-37. Same color scales in the panels could help in the comparisons.
- Supplemental, line 16. “than with COMEPHORE” is maybe “MORDOR-SD”
Citation: https://doi.org/10.5194/egusphere-2023-845-RC1 -
AC1: 'Reply on RC1', Guillaume Evin, 04 Sep 2023
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-845/egusphere-2023-845-AC1-supplement.pdf
-
RC2: 'Comment on egusphere-2023-845', Anonymous Referee #2, 04 Jul 2023
This very well written paper addresses the important question of how to model streamflow in small catchments with different precipitation products. Overall, the paper is a bit limited in terms of references to existing literature on the question of how input resolution interacts with model performance. Also, it does not discuss what different strategies actually exist to infer the hydro model meteorology at the appropriate resolution, from a meterological product that has a different resolution and a different model topography. It appears to me that an essential modelling choice is missing: how to combine the meteo product with the model? There is one option presented per hydro model but we do not know if this is a heuristic choice or the best option or what the literatur says about this. In general, I do e.g. not think that retaining simply the meteo pixels within a catchment is the best option (but perhaps it is for rainfall?). There is a short discussion on the absence of a precip gradient for one product but perhaps it would be good to have a more systematic discussion of how to create the hydro model meteo based on the input meteo.
Also, I strongly recommend to make much clearer what take-aways are relevant beyond the studied catchments and how new they are (at the moment, there are two take-aways which do not reveal completely new insights). And: what does all this for the modelling of ungauged catchments, for which no correction parameters can be calibrated? Would be cool to have some input on this question.
Detailed comments:
- From the abstract, the actual innovation is not clear, it seems like another paper on a previously often studied topic; what is small?
- Very few references at the start of the intro, little reference to extensive literature on the role of spatial resolution of input (rainfall) forcing on the quality of hydrological simulations
- From the methods, I understand that the spatial input product is aggregated to the hydro model by taking the meteo values per pixel: is this a good strategy given that the meteo product does not represent the true topography of the catchment ? In particular for the model that uses elevation bands? How are the pixel of the meteo product matched with the pixels of the hydro model for the other model?
- From the methods it seems like the two models are not calibrated with the same criterion, why? Is this a good idea? does this impact the analysis beyond what is mentioned in the paper?
- The paper would benefit from a concise summary of how the two models differ in terms of process representation? How do they estimate evaporation? Evaporation is almost not mentioned in the entire paper? But it should have a key influence on the representation of the water balance?
- Section 3: how do the model topographies of the meteo products differ?
- Section 4: do I understand correctly that the models are calibrated with criteria that are discussed in the sections dedicated to the models and that other criteria are used to evaluate the performance? Or are the criteria in the model section not relevant? Are the models calibrated with each meteo product? And why is the error on the floods not simply computed as a square-error, is NSE appropriate for this kind of signal? Are the values comparable to those of an entire year?
- Results: are the mNSE values a priori comparable for the different catchments? Since we do not see any streamflow time series and do not know if there are differences in the regimes, it is hard to judge
- please also see a few comments in the annotated pdf
-
AC3: 'Reply on RC2', Guillaume Evin, 04 Sep 2023
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-845/egusphere-2023-845-AC3-supplement.pdf
-
RC3: 'Comment on egusphere-2023-845', Anonymous Referee #3, 23 Jul 2023
The manuscript by Evin et al., assess the performance of different precipitation products in 55 mountainous basins in France, with a specific focus on flood. They conclude that radar measurements are helpful to capture finer scale events leading to flooding, but mountain precipitation is not well captured with radar. This is a clear and straightforward case study that provide incremental insights on precipitation products in mountainous basin. I have a few minor comments below:
I am a bit concerned with the scope of the objective to “choose the best product”. This is very limiting as an objective as it has very limited application for a wider audience. I would reword this objective to something more applicable to a wider range of studies. I suggest shifting the focus of the paper to be about the value added of radar information in mountain basins, as showcased by the analysis of the 55 basins, instead of having a primary objective to “select the best product”.
In the introduction, I would like to see a more robust presentation of radar measurements for precipitation in mountain regions, specifically to how it performs with snow measurements.
I found the most interesting part of the paper to be the discussion. Specifically, the analysis of model performance with different products is linked to process representation in the model (groundwater loss) and precipitation and elevation representation. I would like to see some more information on radar performance for snow vs. rain at higher elevations, and if that could cause some of these issues. Fig 8b also suggests that ERA is actually quite good at capturing high-elevation precipitation, which is a strength that could be mentioned in the conclusion. The groundwater section could also be clearer: Do you mean water is exiting the basin as groundwater, so you have to reduce precipitation? It would be interesting to have more information on how this lack of process representation could be fixed, and what would be advantages of using a model with groundwater processes included in the study.
Citation: https://doi.org/10.5194/egusphere-2023-845-RC3 -
AC4: 'Reply on RC3', Guillaume Evin, 04 Sep 2023
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-845/egusphere-2023-845-AC4-supplement.pdf
-
AC4: 'Reply on RC3', Guillaume Evin, 04 Sep 2023
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 343 | 167 | 33 | 543 | 51 | 14 | 14 |
- HTML: 343
- PDF: 167
- XML: 33
- Total: 543
- Supplement: 51
- BibTeX: 14
- EndNote: 14
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Matthieu Le Lay
Catherine Fouchier
Alexandre Mas
François Colleoni
David Penot
Pierre-André Garambois
Olivier Laurantin
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2701 KB) - Metadata XML
-
Supplement
(10151 KB) - BibTeX
- EndNote
- Final revised paper