the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 18 Oct 2023
Evaluating an Earth system model from a water user perspective
Abstract. The large spatial scale of global Earth system models (ESM) is often cited as an obstacle to using the output by water resource managers in localized decisions. Recent advances in computing have improved the fidelity of hydrological responses in ESMs through increased connectivity between model components. However, the models are seldom evaluated for their ability to reproduce metrics that are important for practitioners, or present the results in a manner that resonates with the users. We draw on the combined experience of the author team and stakeholder workshop participants to identify salient water resource metrics and evaluate whether they are credibly reproduced over the conterminous U.S. by the Community Earth System Model v2 Large Ensemble (CESM2). We find that while the exact values may not match observations, aspects such as interannual variability can be reproduced by CESM2 for the mean wet day precipitation and length of dry spells. CESM2 also captures the proportion of annual total precipitation that derives from the heaviest rain days in watersheds that are not snow-dominated. Aggregating the 7-day mean daily runoff to the watershed scale also shows rain-dominated regions capture the timing and interannual variability in annual maximum and minimum flows. We conclude there is potential for far greater use of large ensemble ESMs, such as CESM2, in long-range water resource decisions to supplement high resolution regional projections.
- Preprint
(2747 KB) - Metadata XML
- BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on egusphere-2023-2326', Anonymous Referee #1, 06 Dec 2023
The authors evaluate the performance of an earth system model (ESM), CESM2, in terms of a set of water availability metrics that support decision making. Here they focus on rainfall and runoff metrics. They found that, although the 100km resolution ESM may not match observations closely, it produces plausible and useful metrics for decision makers. This is from a very interesting perspective, i.e., from a water user perspective. However, the quality of the presentation needs to be improved.
As a person who is not familiar to CESM2, I would appreciate the authors could provide more information regarding the model, like a diagram of the model structure in the appendix, in addition to the sentences at the beginning of Section 3.1 and the reference to the model.
Since CESM2 still perform poorly on some metrics such as WDV and some regions including snow-dominated and mountainous regions, can the authors make some suggestions on how the model could be improved in the future?
Here the authors focus on the HUC2 regions. I am worried that the scale might be too big for local decision makers. Why not using a smaller HUC, e.g., HUC4?
For evaluation, the authors use VIC outputs here, which are model results. Is there a plan to compare with in-situ runoff observations in the future?
Why do the authors use SSP2-4.5 here, not a SSP showing severe climate changes?
Specific comments:
Line 115: Which users here? Do the authors mean the model users?
Line 116: I think the bracket is in the wrong location, and it should be “(N95)”.
Line 213-215: It is hard to observe the similar annual variability in WDV between CESM2 and Livneh from Fig 3cd. Maybe show relative values.
Line 403: What is the higher resolution here?
Citation: https://doi.org/10.5194/egusphere-2023-2326-RC1 -
AC2: 'Reply on RC1', Mari Tye, 25 Mar 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-2326/egusphere-2023-2326-AC2-supplement.pdf
-
AC2: 'Reply on RC1', Mari Tye, 25 Mar 2024
-
CC1: 'Comment on egusphere-2023-2326', Sivarajah Mylevaganam, 20 Feb 2024
Sivarajah Mylevaganam
Alumnus, Spatial Sciences Laboratory, Texas A&M University, College Station, USA.Often times, what we have is insufficient to meet what we need. There are times, what we have is more than what we need. Therefore, to bridge the gap that has been created between what we have and what we need, the principle of sustainable solutions has been preferred in the scientific field. This principle has been well accepted in the fields of economics and business development through the concepts of demand and supply curves.
In this manuscript, considering the spatial resolution and the obstacles introduced by global Earth System Models (ESM), the authors research whether what has been produced through ESMs is useful to meet every local-scale objective and need that is set by practitioners by means of metrics. The findings of the research reveal that some of the metrics that are set by local-scale objectives and needs are well produced by an ESM called the Community Earth System Model (CESM). Therefore, the authors request that we bridge the gap that has been created between what we have through ECMs and what we need through continued collaboration among all stakeholders.
- The title of the manuscript needs to be evaluated by a specialist. In my opinion, these metrics that are evaluated against the model outcome are from practitioners and water managers. These metrics may not reflect what is expected from a water user. This could be well explained if we consider a river basin (e.g., the Mekong River Basin) that is pronounced for upstream-downstream conflicts. The metrics that would be desired by downstream users may not be favored by upstream users. Therefore, policymakers and practitioners decide metrics based on what is best to satisfy both parties (i.e., upstream and downstream users).
- Line 85-88 (Given that ESMs have advanced immeasurably in the recent decade, it is time to re-evaluate whether their direct output can support decision maker)
It would be more appropriate for the authors to enumerate all the advancements in the model to understand these statements.
- The table that has been presented in Appendix A is the culmination point of this research work .In my opinion, the authors need to add more information to understand the necessity of those metrics tabled by the practitioners. For example, as per the table, the number of wet days (NWD) is considered an important metric in reservoir operations management. The inclusion of an exact reason in this table would boost the contribution of this manuscript.
- Refer to Appendix A
Mean precipitation on wet days calculated from PRCPTOT/NWD. Is this correct? As per the definition of PRCPTOT, it includes <1mm of precipitation as well.
- Refer to Part II
Acknowledgement and Disclaimer
The author is an alumnus of Texas A&M University, Texas, USA. The views expressed here are solely those of the author in his private capacity and do not in any way represent the views of Texas A&M University, Texas, USA.
-
AC1: 'Reply on CC1', Mari Tye, 25 Mar 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-2326/egusphere-2023-2326-AC1-supplement.pdf
-
RC2: 'Comment on egusphere-2023-2326', Anonymous Referee #2, 23 Feb 2024
Summary:
This paper provides a detailed evaluation of output from an earth system model, CESM2 over the conterminous US. The authors incorporate some feedback from climate data users to help determine metrics that might be useful, and then determine whether CESM2 can simulate the variables with enough accuracy to be useful without postprocessing of the output. Doing this sort of analysis with the interests of end users of the information at the center adds a useful perspective. While the research presented in this paper is interesting and generally well done, a couple of major shortcomings should be addressed. First, by validating CESM2 in many ways and finding some regions and climatic zones where the output reasonably represents observations, that seems like a verification that CESM2 is a strong candidate for downscaling (like the screening done by Goldenson et al., BAMS 2023, https://doi.org/10.1175/BAMS-D-23-0100.1). While I understand the point of the exercise is to demonstrate the potential value of ESM output without downscaling, if a downscaling method could more closely align output with observed metrics at a finer spatial scale while retaining the large-scale signal from the ESM, it is difficult to see why one would forego downscaling when serving data to stakeholders. The second shortcoming is that, while some skill is clear, especially in rain-dominated basins, it is not shown that broadly averaged statistics at the HUC-2 spatial scale would be actionable information for water managers.
Specific comments:
- Line 24, the “watershed scale” is mentioned, but the scales used in this study would be more accurately described as “continental” or maybe “regional.”
- Section 3.2, the observations are the widely used Livneh data. While the potential to use other data sets is mentioned at the end of the paper, it should be noted that Pierce et al. (JHM 2021, https://doi.org/10.1175/JHM-D-20-0212.1) found biases in extremes in the Livneh data set that produce large discrepancies in extreme precipitation statistics and runoff. A revised version of the Livneh data set is available.
- Lines 174 and 183, For precipitation quantiles and 7Q10 and 7Q90 was a distribution assumed when calculating these probabilities?
- Figures 3 and 5, the gray shading for all ensemble members is very faint, and invisible when printed. It would be better to use the slightly darker shading with the dashed boundaries, as in Figs 6 and 7. As noted in lines 217-220, decision-makers prefer to see individual ensemble members rather than the mean, so maybe that should be done in Fig 3 (as is already done in Fig 5).
- Lines 225-228, It is not clear why poor elevation representation at large scales has caused issues in interannual variability in the South Atlantic and Gulf basins while the mountainous Upper CO and other areas with greater elevation variation apparently have reasonable representation. A plot of elevation variability versus error might strengthen this argument.
- Line 240, a minor point is that the drizzle issue has been known for a long time, so citing an earlier reference (Dai, J. Climate, 2006, Chen et al., J. Climate, 1996, …) would acknowledge that.
- Line 254, this sounds like circular logic, where you determine the precipitation that is exceeded 5% or 1% of the time, and then calculate the frequency of occurrence of those events. The contribution of those events to the annual total precipitation is more meaningful.
- Lines 283-284, the first sentence of the paragraph can be deleted. It is an ordinary change in units that does not need elaboration.
- Lines 309-310, this largely repeats what was stated around line 218, and does not need to be included here.
- Lines 314-316, the argument that in some places the biases in CESM2 may decrease as the climate shifts does not seem like something that would be helpful to decision-makers. For example, if errors in CESM2 are due to failure to represent orographic effects, then a warming atmosphere could exacerbate them.
- Line 334, please do not use parenthetical opposites like “low (high)” (Robock, 2010, doi:10.1029/2010EO450004)
- Table 1: Why are some values in italics?
- Line 349, “our analysis indicates the land surface model correctly simulates the bulk water budget” is not clearly supported from Table 1. It appears only 4 or 5 (depending on the metric) of the 18 HUC-2 basins are statistically similar to the Livneh data. What is the threshold for being “correctly simulated”?
- Line 350, “The projected runoff responses in the regions that will have little to no snow in the future are, therefore, credible” is also not well supported. In fact, the following sentence admits that. As noted in comment 10 above, if errors are due to failure to simulate orographic precipitation because of poor terrain resolution in CESM2, then biases could plausibly increase in the future. The conclusion here seems to be that the runoff does not match Livneh in most locations and that more work is needed to determine why.
- Line 360, which are the “nine regions where CESM2 is credible”? In Table 1 it looks like only two basins have statistically significant correspondence with the Livneh data for all metrics.
- Lines 368-370 largely repeats the figure caption, so is not necessary.
- Lines 382-383 also repeat the figure caption
- Lines 396-397, “While not all of these changes are statistically significant, they are consistent with results…” Changes that are statistically insignificant are indistinguishable from noise, so should not be the basis for drawing conclusions. Restrict the comparison of trends to those locations where significant changes were found.
- Lines 408-409 “CESM2 projects QMax will occur around 5 days earlier in …California by 2020”. As an example of one of my more significant concerns noted in the summary above, it is worth referring to Stewart et al. (Climatic Change, 2004) to appreciate the wide variability in changes in runoff timing across California, both historically and for future projections. It is hard to imagine a water manager making much use of a single projection of a change for all of CA (or HUC 18), even if CESM2 simulates it with some skill. Is there evidence of managers taking action based on this scale of information?
- Table 2 and Figure 10, what is the statistical significance of the projected changes (relative to no change)?
- Lines 461-462, that some aspects of CESM2 precipitation and runoff are “sufficiently credible to support decision needs” or that these results are “plausible enough to support planning around flood and drought control…” was not convincingly demonstrated.
Typos:
- Line 100: “lead” should be “led”
- Line 133: “In part…” is a fragment that should be connected to the prior sentence.
- Line 179, “area of to” is an error
- Line 355, “well-capture” should be “well-captured”
Citation: https://doi.org/10.5194/egusphere-2023-2326-RC2 -
AC2: 'Reply on RC1', Mari Tye, 25 Mar 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-2326/egusphere-2023-2326-AC2-supplement.pdf
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 261 | 94 | 25 | 380 | 15 | 15 |
- HTML: 261
- PDF: 94
- XML: 25
- Total: 380
- BibTeX: 15
- EndNote: 15
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1