Evaluating downscaled products with expected hydroclimatic co-variances

Baek, Seung H.; Ullrich, Paul A.; Dong, Bo; Lee, Jiwoo

doi:https://doi.org/10.5194/egusphere-2024-1456

Preprints

https://doi.org/10.5194/egusphere-2024-1456

Preprints

05 Jun 2024

| 05 Jun 2024

Evaluating downscaled products with expected hydroclimatic co-variances

Seung H. Baek, Paul A. Ullrich, Bo Dong, and Jiwoo Lee

Abstract. There has been widespread adoption of downscaled products amongst practitioners and stakeholders to ascertain risk from climate hazards at the local scale. Such products must nevertheless be consistent with physical laws to be credible and of value to users. Here we evaluate statistically and dynamically downscaled products by examining locally relevant covariances between downscaled temperature and precipitation during convective and frontal precipitation events. We find that two widely-used statistical downscaling techniques (LOCalized Analogs version 2 (LOCA2) and Seasonal Trends and Analysis of Residuals Empirical-Statistical Downscaling Model (STAR-ESDM)) generally preserve expected covariances during convective precipitation events over the historical and future projected intervals. However, both techniques dampen future intensification of frontal precipitation that is otherwise robustly captured in global climate models (i.e., prior to downscaling) and with dynamical downscaling. In the case of LOCA2, this leads to appreciable underestimation of future frontal precipitation events. More broadly, our results suggest that statistical downscaling techniques may be limited in their ability to resolve non-stationary hydrologic processes as compared to dynamical downscaling. Finally, our work proposes expected covariances during convective and frontal precipitation as useful evaluation diagnoses that can be applied universally to a wide range of statistically downscaled products.

Received: 15 May 2024 – Discussion started: 05 Jun 2024

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.

Download & links

Preprint (PDF, 3195 KB)

Supplement (1082 KB)

Download & links

Seung H. Baek, Paul A. Ullrich, Bo Dong, and Jiwoo Lee

Status: closed

RC1:
'Comment on egusphere-2024-1456', Anonymous Referee #1, 04 Jul 2024
The paper deals with the evaluation of statistical and dynamical downscaling of the outputs of global climate models. The goals stated in the introduction are ambitious and interesting; however, the methods used have certain caveats, and the results do not bring any new findings, and the goals are not achieved. I recommend the rejection of the manuscript and encourage resubmission after the following comments are taken into account and the methodology is improved.
More detailed comments:
There are only a few references to related work (e.g., regarding uncertainties related to downscaling methods, evaluation of covariance structure in downscaled products, etc.), and the results obtained are not compared to previous studies.

The definitions of convective and frontal precipitation are rather simplistic. Only one event per year is selected, so only 21 days of each year are used for the analysis. This leads to only a limited amount of data analyzed. There is no discussion of possible other definitions or examples from the literature. Further, it is not quite clear how the events are selected. If the convective precipitation is defined using the annual maximum of air temperature, is it really the case that in every grid point the annual maximum of air temperature is followed by convective precipitation? Moreover, it is not clear how the "peak day" is chosen; further, "peak day" is only analyzed for observed datasets; it is not discussed whether it differs for the downscaling products and model outputs.

The data choice is not explained—why are only 8 CMIP6 GCMs used? For dynamical downscaling, the CMIP5-driven regional climate models are used, whereas for statistical downscaling, the CMIP6 GCMs are incorporated. In my opinion, the comparison of the results would be more informative if the same GCMs for both approaches were used. Moreover, there is no discussion of the choice of two specific statistical downscaling methods. It is claimed that they are "widely used" (l. 73). However, no references or examples are provided.

The covariance between air temperature and precipitation is discussed, but it is not calculated, or the values are not shown. The results are only shown in graphical form, which avoids quantitative evaluation. Moreover, the definitions of both convective precipitation and frontal precipitation, as used here, include the assumption of a temperature-precipitation relationship, making the results less informative. It would be very beneficial if the authors could come up with any quantitative evaluation of the covariances, enabling comparison of assessed methods in some overview figure/table.

It is not explained why the authors concentrate specifically on frontal and convective precipitation. There are plenty of ways how to analyze the temperature-precipitation relationship, and the arguments for this specific choice should be provided.

The conclusions summarized in the last section are very vague. For example, "statistical downscaling may not capture structural change to meteorological phenomena under non-stationarity" or "the dampening to be a spurious feature ... presumably from historical functional relationship and/or the non-stationarity assumption". One of the goals of the study formulated in the introduction was to study these issues in more detail, so, the conclusions of the study should be much stronger and more concrete.

ERA5 downscaled using dynamical downscaling - the references to NA-CORDEX (i.e., Mearns et al., 2017) nor the link to the NA-CORDEX data archive does not show any information about ERA5-driven simulations. From which source did the authors get the ERA5-driven simulations? The referred NA-CORDEX data include only ERA-Interim driven simulations.

More specific/technical comments:
Figures, Figure captions: the term "composite" is not defined; precipitation anomalies shown in absolute values - this is not common, and the negative precipitation anomalies seem very strange; "MAE" and "SD" are not defined and explained; CONUS domain not defined; Fig. 4 - the parentheses are confusing, the caption needs to be reformulated to be more clear. Fig. 3 - for which dataset is it?
Tables: the list of models should be accompanied by more information, e.g., horizontal resolution of the models, modeling centers, etc.
l. 50-51: extremes are not physical processes
l. 58-62: the credibility of methods and relevancy of outputs are presented here to argue for the importance of physical consistency of climate change projections, even though the relevancy is not really important. The credibility based on physical consistency would be enough to introduce the covariance issue.
Section 2: the observed datasets are referred to in a strange manner (e.g., "Livneh-unsplit" is not explained"); The explanation of the STAR-ESDM algorithm is not clear, mainly the term "dynamic climatology"; The length of the studied periods - 35 years - seems rather strange, is not really common. Further, the fact that the reference period of 1980-2014 includes the years 2006-2014, which belong to the scenario simulation in the case of NA-CORDEX simulations. This should be at least mentioned, even though it presumably does not influence the results much.
l. 105: The spatial resolution of LOCA2 outputs is related to the spatial resolution of the underlying observed dataset, isn't it?
l. 119: "Ground truth" is a strange and inappropriate term. The uncertainties related to reference datasets should be discussed.
l. 130: it is not clear how the information in the sentence "We therefore follow..." is implied from the previous sentence.
Section 3: some of the terms used are confusing and uncommon, not well defined, e.g., "parallel time series", "post dynamical downscaling", "ensemble-mean differences" etc.
Citation: https://doi.org/10.5194/egusphere-2024-1456-RC1
- AC2: 'Reply on RC1', Seung-Hun Baek, 15 Aug 2024
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-1456/egusphere-2024-1456-AC2-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2024-1456-AC2
RC2:
'Review of paper“Evaluating downscaled products with expected hydroclimatic co-variances”', Anonymous Referee #2, 01 Aug 2024
Thank you for inviting me to review the paper: “Evaluating downscaled products with expected hydroclimatic co-variances” by Baek et al.
First, please accept my apologies for the slowness in returning this review.
This paper is important and interesting because rather than just considering climate variables of concern in isolation (e.g., their role in extremes), the manuscript emphasizes generating the correct covariances between quantities when downscaling. On that basis, I hope the manuscript will eventually appear in print in some format.
The two obvious variables needing consistency are temperature and heavy precipitation, as noted in the Abstract.
Below are some suggestions that may help towards creating a revised paper version.
In the Abstract, please state the data used to test the two downscaling methods.
In the Abstract, please also provide a typical spatial scale of the analysis that data and models (e.g., RCMs) can currently achieve.
Additionally, the Abstract needs to be easy for a wide audience to understand. The sentence “…our results suggest that statistical downscaling techniques may be limited in their ability to resolve non-stationary hydrologic processes as compared to dynamical downscaling” is slightly ambiguous. Please make clear in the abstract that the word “dynamical” implies a more process-based approach. And “non-stationary”—is that referring to climate change? An Abstract should be broadly understandable in isolation from reading the full manuscript.
The Introduction is good, as it clearly differentiates between statistical and dynamical downscaling. Around line 54, it might help to give citations to example meteorological datasets that are used at the very fine scale, and of course as validation.
The authors could expand a little more around line 61 regarding how there is no guarantee that statistical relationships between variables identified for current levels of atmospheric greenhouse gases will remain valid for future higher GHG concentrations. However a good start is testing performance for current GHG levels.
The paragraph starting “We compare” (line 117) is very helpful and differentiates well between dynamical downscaling (i.e. nested “RCMs”, which contain physical process knowledge that is hopefully also valid for higher GHGs) and inferences from high-resolution meteorological data. The authors make good use of Tables (Table 1, CMIP6 models assessed and Table 2, “data” versus RCM). However, would it be possible to list in a Table, also measurement datasets. A combined Table might help.
A further advantage of a carefully consider Table would be an opportunity to describe all the project names, ESM names, RCM names, datasets. I found myself continuously jumping between diagram captions and different parts of the text to fully understand what I was actually looking at. Some are not even defined – CONUS I guess is C for Canada, US for US? At first, reading the caption to Figure 1, I thought CONUS might be a dataset I had not heard of.
In isolation, Table 2 does not make full sense? If I’ve understood correctly (around line 125), then this Table is which RCMs (top row) are nested in which ESMs (left column). Correct? But I do not fully understand why ERA-Int is mentioned in the left column – surely the comparison would be between ERA-Int and each ESM/RCM combination? Maybe that is what the top row is getting at with “Analyzed” as each entry?
The aspect I like best about the analysis is the ability to compare the first column of Figure 1 (composite behaviour around a convective event) and Figure 2 (composite behaviour around frontal precipitation). To say the obvious, the curve shapes are very different. And critically, the relationship between the T and Precip curves is markedly different between the two. Please make sure that the reader is drawn to these differences in the main writing in the text (and/or discussion).
The next and important comparison, which again needs to come out very clearly from the paper is how actual datasets perform when compared to nested RCMs? Which Figure should I be looking at to compare against Figure1 and 2? Figure 4 is raw CMIP6 GCM data – so not the nested RCM outputs? ESM/RCM combinations are our best estimate of fine-scale meteorological behaviours, and so it should be them that are being tested the most against data?
The main message of the paper is that statistical downscaling does not capture co-variances between P and T as well as ESM/RCM i.e. process (or dynamical) downscaling? So again, the reader needs to see that really clearly too – which I guess implies comparing Figure 1 and 2 against Figure 9?
The reader needs to be steered more clearly, leading to the four main messages:
How do convective P-T relationships differ from frontal P-T relationships?

Do statistical downscaling methods fail at this, as they cannot differentiate between the two necessarily.

Do GCM/RCM combinations capture these differences better? And if so, then:

What do GCM/RCM combinations project into the future?

The presentation is poor in places. For instance, focusing on Figure 5, the left-column x-axis label needs to be marked as “days”. Similarly, units are needed under the colourbars. The maps can be stretched by removing the white space of Canada and Mexico – this will make them easier to interpret. In many places, the diagrams can be enhanced in their appearance.
I am very happy to see the manuscript again, and in the meantime, I hope some of the suggestions and comments above are helpful.
Citation: https://doi.org/10.5194/egusphere-2024-1456-RC2
- AC1: 'Reply on RC2', Seung-Hun Baek, 14 Aug 2024
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-1456/egusphere-2024-1456-AC1-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2024-1456-AC1

Status: closed

RC1:
'Comment on egusphere-2024-1456', Anonymous Referee #1, 04 Jul 2024
The paper deals with the evaluation of statistical and dynamical downscaling of the outputs of global climate models. The goals stated in the introduction are ambitious and interesting; however, the methods used have certain caveats, and the results do not bring any new findings, and the goals are not achieved. I recommend the rejection of the manuscript and encourage resubmission after the following comments are taken into account and the methodology is improved.
More detailed comments:
There are only a few references to related work (e.g., regarding uncertainties related to downscaling methods, evaluation of covariance structure in downscaled products, etc.), and the results obtained are not compared to previous studies.

The definitions of convective and frontal precipitation are rather simplistic. Only one event per year is selected, so only 21 days of each year are used for the analysis. This leads to only a limited amount of data analyzed. There is no discussion of possible other definitions or examples from the literature. Further, it is not quite clear how the events are selected. If the convective precipitation is defined using the annual maximum of air temperature, is it really the case that in every grid point the annual maximum of air temperature is followed by convective precipitation? Moreover, it is not clear how the "peak day" is chosen; further, "peak day" is only analyzed for observed datasets; it is not discussed whether it differs for the downscaling products and model outputs.

The data choice is not explained—why are only 8 CMIP6 GCMs used? For dynamical downscaling, the CMIP5-driven regional climate models are used, whereas for statistical downscaling, the CMIP6 GCMs are incorporated. In my opinion, the comparison of the results would be more informative if the same GCMs for both approaches were used. Moreover, there is no discussion of the choice of two specific statistical downscaling methods. It is claimed that they are "widely used" (l. 73). However, no references or examples are provided.

The covariance between air temperature and precipitation is discussed, but it is not calculated, or the values are not shown. The results are only shown in graphical form, which avoids quantitative evaluation. Moreover, the definitions of both convective precipitation and frontal precipitation, as used here, include the assumption of a temperature-precipitation relationship, making the results less informative. It would be very beneficial if the authors could come up with any quantitative evaluation of the covariances, enabling comparison of assessed methods in some overview figure/table.

It is not explained why the authors concentrate specifically on frontal and convective precipitation. There are plenty of ways how to analyze the temperature-precipitation relationship, and the arguments for this specific choice should be provided.

The conclusions summarized in the last section are very vague. For example, "statistical downscaling may not capture structural change to meteorological phenomena under non-stationarity" or "the dampening to be a spurious feature ... presumably from historical functional relationship and/or the non-stationarity assumption". One of the goals of the study formulated in the introduction was to study these issues in more detail, so, the conclusions of the study should be much stronger and more concrete.

ERA5 downscaled using dynamical downscaling - the references to NA-CORDEX (i.e., Mearns et al., 2017) nor the link to the NA-CORDEX data archive does not show any information about ERA5-driven simulations. From which source did the authors get the ERA5-driven simulations? The referred NA-CORDEX data include only ERA-Interim driven simulations.

More specific/technical comments:
Figures, Figure captions: the term "composite" is not defined; precipitation anomalies shown in absolute values - this is not common, and the negative precipitation anomalies seem very strange; "MAE" and "SD" are not defined and explained; CONUS domain not defined; Fig. 4 - the parentheses are confusing, the caption needs to be reformulated to be more clear. Fig. 3 - for which dataset is it?
Tables: the list of models should be accompanied by more information, e.g., horizontal resolution of the models, modeling centers, etc.
l. 50-51: extremes are not physical processes
l. 58-62: the credibility of methods and relevancy of outputs are presented here to argue for the importance of physical consistency of climate change projections, even though the relevancy is not really important. The credibility based on physical consistency would be enough to introduce the covariance issue.
Section 2: the observed datasets are referred to in a strange manner (e.g., "Livneh-unsplit" is not explained"); The explanation of the STAR-ESDM algorithm is not clear, mainly the term "dynamic climatology"; The length of the studied periods - 35 years - seems rather strange, is not really common. Further, the fact that the reference period of 1980-2014 includes the years 2006-2014, which belong to the scenario simulation in the case of NA-CORDEX simulations. This should be at least mentioned, even though it presumably does not influence the results much.
l. 105: The spatial resolution of LOCA2 outputs is related to the spatial resolution of the underlying observed dataset, isn't it?
l. 119: "Ground truth" is a strange and inappropriate term. The uncertainties related to reference datasets should be discussed.
l. 130: it is not clear how the information in the sentence "We therefore follow..." is implied from the previous sentence.
Section 3: some of the terms used are confusing and uncommon, not well defined, e.g., "parallel time series", "post dynamical downscaling", "ensemble-mean differences" etc.
Citation: https://doi.org/10.5194/egusphere-2024-1456-RC1
- AC2: 'Reply on RC1', Seung-Hun Baek, 15 Aug 2024
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-1456/egusphere-2024-1456-AC2-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2024-1456-AC2
RC2:
'Review of paper“Evaluating downscaled products with expected hydroclimatic co-variances”', Anonymous Referee #2, 01 Aug 2024
Thank you for inviting me to review the paper: “Evaluating downscaled products with expected hydroclimatic co-variances” by Baek et al.
First, please accept my apologies for the slowness in returning this review.
This paper is important and interesting because rather than just considering climate variables of concern in isolation (e.g., their role in extremes), the manuscript emphasizes generating the correct covariances between quantities when downscaling. On that basis, I hope the manuscript will eventually appear in print in some format.
The two obvious variables needing consistency are temperature and heavy precipitation, as noted in the Abstract.
Below are some suggestions that may help towards creating a revised paper version.
In the Abstract, please state the data used to test the two downscaling methods.
In the Abstract, please also provide a typical spatial scale of the analysis that data and models (e.g., RCMs) can currently achieve.
Additionally, the Abstract needs to be easy for a wide audience to understand. The sentence “…our results suggest that statistical downscaling techniques may be limited in their ability to resolve non-stationary hydrologic processes as compared to dynamical downscaling” is slightly ambiguous. Please make clear in the abstract that the word “dynamical” implies a more process-based approach. And “non-stationary”—is that referring to climate change? An Abstract should be broadly understandable in isolation from reading the full manuscript.
The Introduction is good, as it clearly differentiates between statistical and dynamical downscaling. Around line 54, it might help to give citations to example meteorological datasets that are used at the very fine scale, and of course as validation.
The authors could expand a little more around line 61 regarding how there is no guarantee that statistical relationships between variables identified for current levels of atmospheric greenhouse gases will remain valid for future higher GHG concentrations. However a good start is testing performance for current GHG levels.
The paragraph starting “We compare” (line 117) is very helpful and differentiates well between dynamical downscaling (i.e. nested “RCMs”, which contain physical process knowledge that is hopefully also valid for higher GHGs) and inferences from high-resolution meteorological data. The authors make good use of Tables (Table 1, CMIP6 models assessed and Table 2, “data” versus RCM). However, would it be possible to list in a Table, also measurement datasets. A combined Table might help.
A further advantage of a carefully consider Table would be an opportunity to describe all the project names, ESM names, RCM names, datasets. I found myself continuously jumping between diagram captions and different parts of the text to fully understand what I was actually looking at. Some are not even defined – CONUS I guess is C for Canada, US for US? At first, reading the caption to Figure 1, I thought CONUS might be a dataset I had not heard of.
In isolation, Table 2 does not make full sense? If I’ve understood correctly (around line 125), then this Table is which RCMs (top row) are nested in which ESMs (left column). Correct? But I do not fully understand why ERA-Int is mentioned in the left column – surely the comparison would be between ERA-Int and each ESM/RCM combination? Maybe that is what the top row is getting at with “Analyzed” as each entry?
The aspect I like best about the analysis is the ability to compare the first column of Figure 1 (composite behaviour around a convective event) and Figure 2 (composite behaviour around frontal precipitation). To say the obvious, the curve shapes are very different. And critically, the relationship between the T and Precip curves is markedly different between the two. Please make sure that the reader is drawn to these differences in the main writing in the text (and/or discussion).
The next and important comparison, which again needs to come out very clearly from the paper is how actual datasets perform when compared to nested RCMs? Which Figure should I be looking at to compare against Figure1 and 2? Figure 4 is raw CMIP6 GCM data – so not the nested RCM outputs? ESM/RCM combinations are our best estimate of fine-scale meteorological behaviours, and so it should be them that are being tested the most against data?
The main message of the paper is that statistical downscaling does not capture co-variances between P and T as well as ESM/RCM i.e. process (or dynamical) downscaling? So again, the reader needs to see that really clearly too – which I guess implies comparing Figure 1 and 2 against Figure 9?
The reader needs to be steered more clearly, leading to the four main messages:
How do convective P-T relationships differ from frontal P-T relationships?

Do statistical downscaling methods fail at this, as they cannot differentiate between the two necessarily.

Do GCM/RCM combinations capture these differences better? And if so, then:

What do GCM/RCM combinations project into the future?

The presentation is poor in places. For instance, focusing on Figure 5, the left-column x-axis label needs to be marked as “days”. Similarly, units are needed under the colourbars. The maps can be stretched by removing the white space of Canada and Mexico – this will make them easier to interpret. In many places, the diagrams can be enhanced in their appearance.
I am very happy to see the manuscript again, and in the meantime, I hope some of the suggestions and comments above are helpful.
Citation: https://doi.org/10.5194/egusphere-2024-1456-RC2
- AC1: 'Reply on RC2', Seung-Hun Baek, 14 Aug 2024
  
  The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-1456/egusphere-2024-1456-AC1-supplement.pdf
  
  Citation: https://doi.org/10.5194/egusphere-2024-1456-AC1

Seung H. Baek, Paul A. Ullrich, Bo Dong, and Jiwoo Lee

Supplement

https://doi.org/10.5194/egusphere-2024-1456-supplement

Seung H. Baek, Paul A. Ullrich, Bo Dong, and Jiwoo Lee

Viewed

Total article views: 511 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	Supplement	BibTeX	EndNote
382	98	31	511	34	15	14

HTML: 382
PDF: 98
XML: 31
Total: 511
Supplement: 34
BibTeX: 15
EndNote: 14

Views and downloads (calculated since 05 Jun 2024)

Month	HTML	PDF	XML	Total
Jun 2024	127	35	11	173
Jul 2024	68	16	9	93
Aug 2024	119	31	8	158
Sep 2024	27	6	1	34
Oct 2024	20	6	2	28
Nov 2024	21	4	0	25

Cumulative views and downloads (calculated since 05 Jun 2024)

Month	HTML	PDF	XML	Total
Jun 2024	127	35	11	173
Jul 2024	68	16	9	93
Aug 2024	119	31	8	158
Sep 2024	27	6	1	34
Oct 2024	20	6	2	28
Nov 2024	21	4	0	25

Viewed (geographical distribution)

Total article views: 498 (including HTML, PDF, and XML) Thereof 498 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 20 Nov 2024

This paper addresses the conditions in which GCM and downscaled solutions diverge for targeted processes under historical and future climate conditions. Downscaling is a crucial part of making climate model outputs useable by the wider science and policy community. Understanding the properties and limitations of downscaling should hence be of interest far beyond the model development community.

Short summary

We evaluate downscaled products by examining locally relevant covariances during convective and frontal precipitation events. Common statistical downscaling techniques preserve expected covariances during convective precipitation. However, they dampen future intensification of frontal precipitation captured in global climate models and dynamical downscaling. This suggests statistical downscaling may not fully resolve non-stationary hydrologic processes as compared to dynamical downscaling.


Total:	0
HTML:	0
PDF:	0
XML:	0