the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 08 Aug 2024
Learning from a large-scale calibration effort of multiple lake models
Abstract. Process-based hydrodynamic lake models are commonly used for simulating water temperature, enabling testing of different scenarios and drawing conclusions about possible water quality developments or changes in important ecological processes such as methane gas emissions. Even though there are several models available, a systematic comparison regarding their performance is missing so far. In this study, we calibrated four different one-dimensional hydrodynamic lake models for a global dataset of 73 lakes to compare their performance in reproducing water temperature and estimated parameter sensitivity for the calibrated parameters. The models performance and parameter sensitivity showed a relation to the lake characteristics and model structure. No single model was the best, with each model performing better than the rest in at least some of the lakes. From the findings, we advocate the application of model ensembles. Nonetheless, we also highlight the need to further improve both weather forcing data, individual models, and multi-model ensemble techniques.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(1119 KB)
-
Supplement
(1303 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(1119 KB) - Metadata XML
-
Supplement
(1303 KB) - BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
CC1: 'Comment on egusphere-2024-2447', John Ding, 10 Aug 2024
Figure 2. Distribution of the six evaluated performance metrics
In Figure 2, the histograms of the NSE (Nash-Sutcliffe efficiency) and R (Pearson correlation coefficient) highlight calibration results of four lake models on a daily step. The former is a variance-based, and the latter, a correlation-based metric. The simulated data counts cluster around a median value of 0.96 (an exact value not shown in the text) and of 0.98 (shown in Line 220), respectively, both at the upper end of their performance scale.
In a different context of rainfall-runoff modelling, for an observed hydrograph, Duc and Sawada (2023, Equation 25, Figure 2) show that the upper end/bound of the NSE is related to the correlation coefficient, R, as follows: NSEu=2-1/(RxR).
The median NSE and R values for simulated lake water temperatures appear to follow the equality.
Reference
Duc, L. and Sawada, Y.: A signal-processing-based interpretation of the Nash–Sutcliffe efficiency, Hydrol. Earth Syst. Sci., 27, 1827–1839, https://doi.org/10.5194/hess-27-1827-2023, 2023.
Citation: https://doi.org/10.5194/egusphere-2024-2447-CC1 -
AC1: 'Reply on CC1', Johannes Feldbauer, 30 Oct 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-2447/egusphere-2024-2447-AC1-supplement.pdf
-
AC1: 'Reply on CC1', Johannes Feldbauer, 30 Oct 2024
-
RC1: 'Comment on egusphere-2024-2447', Zeli Tan, 02 Sep 2024
Feldbauer et al. present a timely and very interesting study related to the uncertainty of physical lake models. Given the critical role of the studied models in climate impact analysis, such as ISIMIP, this research provides important implications for a better assessment of global change in lakes and reservoirs. The manuscript is with high quality of presentation and rigorous scientific inquiry. Only some clarifications and extended discussions are needed before it can be accepted for publication.
One limitation of the current approach, which simultaneously assesses three types of uncertainties in lake models (i.e., input uncertainty, parameter uncertainty, and structural uncertainty), should be further discussed. Due to the interactions among these uncertainties, it is challenging for this multi-objective approach to fully resolve specific uncertainties. As admitted in the manuscript, the uncertainty of the three input-related scaling factors may hide the uncertainty of model-specific parameters and process descriptions. Consequently, the method obscures the investigation of an optimal model for specific lake types and an optimal algorithm for specific lake processes. Notably, for global lake modeling, we hope that as climate data become more accurate with time, the uncertainty of global lake simulations will be reduced. We also hope that lake models can achieve consistent global simulations without lake-specific calibrations. Despite the great values of the current paper, it falls short in addressing these issues. Conversely, I would encourage the authors to conduct a future study which uses observed atmospheric forcings to exclude the uncertainty in input data and ensure the good model performance achieved for right reasons. There are some existing studies following the recommended approach, such as Guseva et al. (2020) and Guo et al. (2021). But their values are limited due to either focusing on only one lake or testing only one lake model.
Guo, M., Zhuang, Q., Yao, H., Golub, M., Leung, L. R., & Tan, Z. (2021). Intercomparison of thermal regime algorithms in 1-D lake models. Water Resources Research, 57, e2020WR028776. https://doi. org/10.1029/2020WR028776
Guseva, S., Bleninger, T., Jöhnk, K., Polli, B. A., Tan, Z., Thiery, W., ... & Stepanenko, V. (2020). Multimodel simulation of vertical gas transfer in a temperate lake. Hydrology and Earth System Sciences, 24(2), 697-715.
I suggest the authors to avoid the use of "hydrodynamic lake models" to describe the studied models. To me, hydrodynamic models refer to numerical models that solve the transport of both mass and momentum. The authors can use "physical lake models", "thermodynamic lake models", or just "lake models".
In the methodology, one area that needs clarification is what procedure the authors have adopted to ensure appropriate initial conditions for simulations. It can be particularly important for the modeling of deep lakes.
Minor comments:
L55: also Zhuang et al. (2023). Zhuang, Q., Guo, M., Melack, J. M., Lan, X., Tan, Z., Oh, Y., & Leung, L. R. (2023). Current and future global lake methane emissions: A process-based modeling analysis. Journal of Geophysical Research: Biogeosciences, 128, e2022JG007137. https://doi. org/10.1029/2022JG007137
L181: gets absorbed
L202: How does the metric δ differ from the Sobol's total-order index?
L212: What is "SA"?
L226-227: I suggest moving Figure 3 upward to Section 2.2
L232: Figure S5 is introduced prior to that of Figure S4.
Figure 5: It is surprising to see that S1 is larger than δ in many cases. To my experience, the first-order sensitivity should be smaller than the total-order sensitivity.
L306: remove "a"
L407: potentially
Citation: https://doi.org/10.5194/egusphere-2024-2447-RC1 -
AC2: 'Reply on RC1', Johannes Feldbauer, 30 Oct 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-2447/egusphere-2024-2447-AC2-supplement.pdf
-
AC2: 'Reply on RC1', Johannes Feldbauer, 30 Oct 2024
-
RC2: 'Comment on egusphere-2024-2447', Fabian Bärenbold, 23 Sep 2024
General comments
One-dimensional physical lake models are a widely used tool in simulations of lakes and reservoirs for diverse goals like now-casting, forecasting, or to compute mixing for use biogeochemistry. Although many different models exist and several of them are very widely used, to my knowledge no consistent comparison between them exists until now on a wide range of lakes. In general, research about the link between lake types, model parameters and uncertainty is not widespread and I think this manuscript is a good contribution to fill this gap. The manuscript is well organized and written with only few spelling mistakes. However, I think that there is a problem with one of the calibrated parameters of GOTM and not enough details on the observational data.
Specific comments:
The observational data used to calibrate the 73 lake models is of great importance, but very few details are given in this publication. I would find it useful to a small paragraph about time and depth resolution of the observational data and whether any weighting was done to compute performance metrics (lines 196 – 198). Is there a minimum requirement of observations to be included to the 73 lakes? In addition, I would welcome a Figure giving some information about depth and time resolution of measurements in the supporting information.
Is there a reason the GSWP3-W5E5 reanalysis dataset was chosen? Could you explain this in 1 – 2 sentences? Also a bit related, is there any chance of going to hourly instead of daily resolution? As far as I understand from the discussion this might solve some of the problems (wind speed effect on mixing is cubic). If yes, it would be interesting to mention this in the discussion/conclusion.
I would find it interesting to have the silhouette plot of the cluster analysis in the supporting information.
The authors discuss the fact that k_min of GOTM seems to be the most sensitive of the lake-specific calibration parameters. The range of k_min used in this study is 1.5e-7 to 1e-5, which is rather high values compared to a default value of 1e-8 in the GOTM manual. In addition, typical values of ~1e-6 are reported for TKE in the hypolimnion of lakes (e.g. Wüest and Lorke, 2003). I see a major problem if k_min is set too high: it could, together with high values for the scaling of shortwave radiation, offset a low value for the scaling of wind speed. There is evidence of this in Figure 7, where the calibrated parameters for GOTM are consistently low (wind speed) or high (shortwave) compared to the other models. The calibrated value of k_min always seems to be well above 1e-6 (Figure S8), so on the order of typically observed values for TKE (Wüest and Lorke, 2003). The authors discuss the potential interactions between k_min and wind scaling on one hand (lines 341 – 345), and shortwave scaling and wind scaling on the other hand (lines 365 – 369) but not the potential interaction between all 3. In regard of this, I would like to ask the authors to motivate their choice of the range of k_min and to check its influence on the wind and shortwave scaling parameters. I see two ways to do this:
- Compute second/third order Sobol indices for the concerned parameters
- Redo some of the simulations with k_min = 1e-8
I understand that the interaction calculation shows that this suggested parameter interaction is not driving variance but the correlation could still be strong among these 3 parameters. There is also no obvious reason why wind and shortwave scaling should be so different for similar models like Simstrat and GOTM. I could be wrong but to me it seems like k_min in GOTM is playing the role of the seiche in Simstrat.
The authors seem to imply that larger lakes should generally have a larger value for wind scaling and vice versa for small lakes (lines 349 – 351). However, if true this effect should be visible in the calibrated wind scaling parameters, no?
Technical comments:
Title: maybe “hydrodynamic” or “physical” lake models instead of just lake models
Line 6: The following sentence is a bit too unspecific to me: “The models performance and parameter sensitivity showed a relation to the lake characteristics and model structure.”
Line 39 - 40: Maybe mention that although important for shallow lakes, the biogeochemical components are not discussed in this manuscript.
Line 64: Make clear that Table S1 of the current manuscript is meant and not Table S1 of Golub et al. (2022)
Line 82: delete “and” before “the 1D turbulence-based models GOTM”
Line 99: “comma” after “Equation 1”
Equation 3: Shouldn’t it be h < z < D_lake?
Equation 5: Please check whether term 1 is really negative.
Line 220: This sentence is not clear to me. Did the authors intend to say “between best and worst performing model”?
Line 263: “were” instead of “was”
Line 269: “Lake clusters” instead of “lake cluster”
Line 357: “better” instead of “more”
Citation: https://doi.org/10.5194/egusphere-2024-2447-RC2 -
AC3: 'Reply on RC2', Johannes Feldbauer, 30 Oct 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-2447/egusphere-2024-2447-AC3-supplement.pdf
Interactive discussion
Status: closed
-
CC1: 'Comment on egusphere-2024-2447', John Ding, 10 Aug 2024
Figure 2. Distribution of the six evaluated performance metrics
In Figure 2, the histograms of the NSE (Nash-Sutcliffe efficiency) and R (Pearson correlation coefficient) highlight calibration results of four lake models on a daily step. The former is a variance-based, and the latter, a correlation-based metric. The simulated data counts cluster around a median value of 0.96 (an exact value not shown in the text) and of 0.98 (shown in Line 220), respectively, both at the upper end of their performance scale.
In a different context of rainfall-runoff modelling, for an observed hydrograph, Duc and Sawada (2023, Equation 25, Figure 2) show that the upper end/bound of the NSE is related to the correlation coefficient, R, as follows: NSEu=2-1/(RxR).
The median NSE and R values for simulated lake water temperatures appear to follow the equality.
Reference
Duc, L. and Sawada, Y.: A signal-processing-based interpretation of the Nash–Sutcliffe efficiency, Hydrol. Earth Syst. Sci., 27, 1827–1839, https://doi.org/10.5194/hess-27-1827-2023, 2023.
Citation: https://doi.org/10.5194/egusphere-2024-2447-CC1 -
AC1: 'Reply on CC1', Johannes Feldbauer, 30 Oct 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-2447/egusphere-2024-2447-AC1-supplement.pdf
-
AC1: 'Reply on CC1', Johannes Feldbauer, 30 Oct 2024
-
RC1: 'Comment on egusphere-2024-2447', Zeli Tan, 02 Sep 2024
Feldbauer et al. present a timely and very interesting study related to the uncertainty of physical lake models. Given the critical role of the studied models in climate impact analysis, such as ISIMIP, this research provides important implications for a better assessment of global change in lakes and reservoirs. The manuscript is with high quality of presentation and rigorous scientific inquiry. Only some clarifications and extended discussions are needed before it can be accepted for publication.
One limitation of the current approach, which simultaneously assesses three types of uncertainties in lake models (i.e., input uncertainty, parameter uncertainty, and structural uncertainty), should be further discussed. Due to the interactions among these uncertainties, it is challenging for this multi-objective approach to fully resolve specific uncertainties. As admitted in the manuscript, the uncertainty of the three input-related scaling factors may hide the uncertainty of model-specific parameters and process descriptions. Consequently, the method obscures the investigation of an optimal model for specific lake types and an optimal algorithm for specific lake processes. Notably, for global lake modeling, we hope that as climate data become more accurate with time, the uncertainty of global lake simulations will be reduced. We also hope that lake models can achieve consistent global simulations without lake-specific calibrations. Despite the great values of the current paper, it falls short in addressing these issues. Conversely, I would encourage the authors to conduct a future study which uses observed atmospheric forcings to exclude the uncertainty in input data and ensure the good model performance achieved for right reasons. There are some existing studies following the recommended approach, such as Guseva et al. (2020) and Guo et al. (2021). But their values are limited due to either focusing on only one lake or testing only one lake model.
Guo, M., Zhuang, Q., Yao, H., Golub, M., Leung, L. R., & Tan, Z. (2021). Intercomparison of thermal regime algorithms in 1-D lake models. Water Resources Research, 57, e2020WR028776. https://doi. org/10.1029/2020WR028776
Guseva, S., Bleninger, T., Jöhnk, K., Polli, B. A., Tan, Z., Thiery, W., ... & Stepanenko, V. (2020). Multimodel simulation of vertical gas transfer in a temperate lake. Hydrology and Earth System Sciences, 24(2), 697-715.
I suggest the authors to avoid the use of "hydrodynamic lake models" to describe the studied models. To me, hydrodynamic models refer to numerical models that solve the transport of both mass and momentum. The authors can use "physical lake models", "thermodynamic lake models", or just "lake models".
In the methodology, one area that needs clarification is what procedure the authors have adopted to ensure appropriate initial conditions for simulations. It can be particularly important for the modeling of deep lakes.
Minor comments:
L55: also Zhuang et al. (2023). Zhuang, Q., Guo, M., Melack, J. M., Lan, X., Tan, Z., Oh, Y., & Leung, L. R. (2023). Current and future global lake methane emissions: A process-based modeling analysis. Journal of Geophysical Research: Biogeosciences, 128, e2022JG007137. https://doi. org/10.1029/2022JG007137
L181: gets absorbed
L202: How does the metric δ differ from the Sobol's total-order index?
L212: What is "SA"?
L226-227: I suggest moving Figure 3 upward to Section 2.2
L232: Figure S5 is introduced prior to that of Figure S4.
Figure 5: It is surprising to see that S1 is larger than δ in many cases. To my experience, the first-order sensitivity should be smaller than the total-order sensitivity.
L306: remove "a"
L407: potentially
Citation: https://doi.org/10.5194/egusphere-2024-2447-RC1 -
AC2: 'Reply on RC1', Johannes Feldbauer, 30 Oct 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-2447/egusphere-2024-2447-AC2-supplement.pdf
-
AC2: 'Reply on RC1', Johannes Feldbauer, 30 Oct 2024
-
RC2: 'Comment on egusphere-2024-2447', Fabian Bärenbold, 23 Sep 2024
General comments
One-dimensional physical lake models are a widely used tool in simulations of lakes and reservoirs for diverse goals like now-casting, forecasting, or to compute mixing for use biogeochemistry. Although many different models exist and several of them are very widely used, to my knowledge no consistent comparison between them exists until now on a wide range of lakes. In general, research about the link between lake types, model parameters and uncertainty is not widespread and I think this manuscript is a good contribution to fill this gap. The manuscript is well organized and written with only few spelling mistakes. However, I think that there is a problem with one of the calibrated parameters of GOTM and not enough details on the observational data.
Specific comments:
The observational data used to calibrate the 73 lake models is of great importance, but very few details are given in this publication. I would find it useful to a small paragraph about time and depth resolution of the observational data and whether any weighting was done to compute performance metrics (lines 196 – 198). Is there a minimum requirement of observations to be included to the 73 lakes? In addition, I would welcome a Figure giving some information about depth and time resolution of measurements in the supporting information.
Is there a reason the GSWP3-W5E5 reanalysis dataset was chosen? Could you explain this in 1 – 2 sentences? Also a bit related, is there any chance of going to hourly instead of daily resolution? As far as I understand from the discussion this might solve some of the problems (wind speed effect on mixing is cubic). If yes, it would be interesting to mention this in the discussion/conclusion.
I would find it interesting to have the silhouette plot of the cluster analysis in the supporting information.
The authors discuss the fact that k_min of GOTM seems to be the most sensitive of the lake-specific calibration parameters. The range of k_min used in this study is 1.5e-7 to 1e-5, which is rather high values compared to a default value of 1e-8 in the GOTM manual. In addition, typical values of ~1e-6 are reported for TKE in the hypolimnion of lakes (e.g. Wüest and Lorke, 2003). I see a major problem if k_min is set too high: it could, together with high values for the scaling of shortwave radiation, offset a low value for the scaling of wind speed. There is evidence of this in Figure 7, where the calibrated parameters for GOTM are consistently low (wind speed) or high (shortwave) compared to the other models. The calibrated value of k_min always seems to be well above 1e-6 (Figure S8), so on the order of typically observed values for TKE (Wüest and Lorke, 2003). The authors discuss the potential interactions between k_min and wind scaling on one hand (lines 341 – 345), and shortwave scaling and wind scaling on the other hand (lines 365 – 369) but not the potential interaction between all 3. In regard of this, I would like to ask the authors to motivate their choice of the range of k_min and to check its influence on the wind and shortwave scaling parameters. I see two ways to do this:
- Compute second/third order Sobol indices for the concerned parameters
- Redo some of the simulations with k_min = 1e-8
I understand that the interaction calculation shows that this suggested parameter interaction is not driving variance but the correlation could still be strong among these 3 parameters. There is also no obvious reason why wind and shortwave scaling should be so different for similar models like Simstrat and GOTM. I could be wrong but to me it seems like k_min in GOTM is playing the role of the seiche in Simstrat.
The authors seem to imply that larger lakes should generally have a larger value for wind scaling and vice versa for small lakes (lines 349 – 351). However, if true this effect should be visible in the calibrated wind scaling parameters, no?
Technical comments:
Title: maybe “hydrodynamic” or “physical” lake models instead of just lake models
Line 6: The following sentence is a bit too unspecific to me: “The models performance and parameter sensitivity showed a relation to the lake characteristics and model structure.”
Line 39 - 40: Maybe mention that although important for shallow lakes, the biogeochemical components are not discussed in this manuscript.
Line 64: Make clear that Table S1 of the current manuscript is meant and not Table S1 of Golub et al. (2022)
Line 82: delete “and” before “the 1D turbulence-based models GOTM”
Line 99: “comma” after “Equation 1”
Equation 3: Shouldn’t it be h < z < D_lake?
Equation 5: Please check whether term 1 is really negative.
Line 220: This sentence is not clear to me. Did the authors intend to say “between best and worst performing model”?
Line 263: “were” instead of “was”
Line 269: “Lake clusters” instead of “lake cluster”
Line 357: “better” instead of “more”
Citation: https://doi.org/10.5194/egusphere-2024-2447-RC2 -
AC3: 'Reply on RC2', Johannes Feldbauer, 30 Oct 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-2447/egusphere-2024-2447-AC3-supplement.pdf
Peer review completion
Journal article(s) based on this preprint
Data sets
Data analysis and plots Johannes Feldbauer and Jorrit P. Mesman https://zenodo.org/doi/10.5281/zenodo.13150422
Set up and run calibration Jorrit P. Mesman and Johannes Feldbauer https://zenodo.org/doi/10.5281/zenodo.13165427
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 404 | 121 | 182 | 707 | 31 | 15 | 18 |
- HTML: 404
- PDF: 121
- XML: 182
- Total: 707
- Supplement: 31
- BibTeX: 15
- EndNote: 18
Viewed (geographical distribution)
| Country | # | Views | % |
|---|---|---|---|
| United States of America | 1 | 211 | 28 |
| Germany | 2 | 78 | 10 |
| Denmark | 3 | 64 | 8 |
| China | 4 | 37 | 4 |
| Switzerland | 5 | 30 | 3 |
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
- 211
Johannes Feldbauer
Jorrit P. Mesman
Tobias K. Andersen
Robert Ladwig
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(1119 KB) - Metadata XML
-
Supplement
(1303 KB) - BibTeX
- EndNote
- Final revised paper