the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 27 Feb 2025
Brief communication: Reanalyses underperform in cold regions, raising concerns for climate services and research
Abstract. Many changes in cold regions are amplified by nonlinear processes involving ice, and have important consequences locally and globally. We show that the average ensemble spread of the mean annual air temperature (1.5 °C) in the reanalyses is 90 % greater in cold regions compared to the other regions and shows pronounced disagreement in the trend. The ensemble spread in the mean annual maximum snow water equivalent is found greater than the ensemble mean. The reduced quality of reanalyses in cold regions, coinciding with sparse in situ observations and low population, points to challenges in how we represent cold-regions phenomena in simulation systems and limits our ability to support climate research and services.
- Preprint
(1742 KB) - Metadata XML
-
Supplement
(1035 KB) - BibTeX
- EndNote
Status: closed
-
CC1: 'Comment on egusphere-2025-575', Steven Margulis, 28 Feb 2025
-
AC2: 'Reply on CC1', Bin Cao, 20 May 2025
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2025/egusphere-2025-575/egusphere-2025-575-AC2-supplement.pdf
-
AC2: 'Reply on CC1', Bin Cao, 20 May 2025
-
RC1: 'Comment on egusphere-2025-575', Anonymous Referee #1, 01 Apr 2025
General Comments
Cao and Gruber investigate the performance of five modern reanalyses (JRA-3Q, ERA5, MERRA-2, JRA-55 and NCEP2) over cold regions, with a focus on air temperature and snow water equivalent (SWE). They show that the ensemble spread in mean annual air temperature (MAAT) in reanalyses is up 90% greater over cold regions (defined as regions with a MAAT <0oC) relative to regions with a MAAT ≥ 0oC. The study explicitly shows the relationship between station density and ensemble spread for both SWE and MAAT and is able to show that the reduced reanalysis performance is at least partially related to the low station density over cold regions.
The bulk of the conclusions come from Figure 1, which show the average ensemble spread in MAAT (Panel A) and SWE (Panel B) binned by MAAT. The station density in each MAAT bin, and the proportion of the grid cells within the MAAT bin covered by ice sheets and glaciers, snow cover, permafrost, and seasonally frozen ground is also shown. Herrington et al. (2024) showed a similar plot for soil temperature, that identified the average reanalysis spread in soil temperatures binned by MAAT, against sample size, though it didn’t explicitly consider station density, or the proportion of the grid cells covered by the cryosphere – so this is a novel analysis (along with the focus on SWE and MAAT).
While Figure 1 clearly shows a clear correlation between ensemble spread, station density, and the presence of cryospheric elements, there is no attempt to separate the contributions of low station density from those related to inadequate representation of cold region processes in reanalyses. As a reader it raises questions as to what the relative contributions of station density, and inadequate process representation to the ensemble spread in MAAT and SWE are?
To me, the novel science is in quantifying what proportion of the uncertainty can be attributed to station density, and what proportion is related to inadequate representation of cold region processes, which have been thoroughly discussed in the literature (e.g. Broxton et al., 2016; Cao et al., 2020, 2022; Hu et al., 2019; Mortimer et al., 2020; Wang et al., 2019). Thus, I recommend that the authors extend their analysis to explicitly quantify what proportion of the uncertainties or spread in MAAT and SWE can be attributed to the low station density in cold regions, and what proportion can be attributed to the inadequate process representation in the products. This will greatly enhance the contribution of the paper to the literature and provide the community with useful and quantifiable estimates of the uncertainty attribution.
Specific Comments
P2, L31: Why was NCEP2 investigated over NCEP CFSR/CFSv2, for example? NCEP CFSR/CFSv2 is available at a much higher resolution than NCEP2, and is available over the period of analysis (1991-2020).
P2, L36-L37: What do the authors mean by the statement “the decades 1991-2020 were used, likely a period of high quality for reanalyses.” Is there a particular standard by which the authors determined this? Some clarification would be helpful.
P2, L38: What is the CDS? It appears that CDS is in brackets, but the acronym is not defined. I presume this may be the Climate Data Store?
P4, L101: The all 5-reanalysis value for MAAT spread was (1.5oC, 0.5oC-3.0oC) - is this statistically different from the value for the 4DVar reanalyses reported here?
P10, Figure 1: What do the dashed lines represent in Figure 1? I don’t see a dashed line in the figure legend?
P11, Figure 2: It may be helpful to have a "difference" panel between the All 5 reanalyses and the three 4DVar reanalyses to highlight the regions with the largest differences, particularly for SWE, since it is a little harder to notice the differences in Panel B.
Technical Comments
P2, L38: replace “for” in “for International Earth Science Information Network…” with “from”.
P2, L81: replace “were” in the sentence “were slightly more permafrost…” with “where”.
References
Broxton, P. D., Zeng, X., and Dawson, N.: Why Do Global Reanalyses and Land Data Assimilation Products Underestimate Snow Water Equivalent?, J. Hydrometeorol., 17, 2743–2761, https://doi.org/10.1175/JHM-D-16-0056.1, 2016.
Cao, B., Gruber, S., Zheng, D., and Li, X.: The ERA5-Land soil temperature bias in permafrost regions, The Cryosphere, 14, 2581–2595, https://doi.org/10.5194/tc-14-2581-2020, 2020.
Cao, B., Arduini, G., and Zsoter, E.: Brief communication: Improving ERA5-Land soil temperature in permafrost regions using an optimized multi-layer snow scheme, The Cryosphere, 16, 2701–2708, https://doi.org/10.5194/tc-16-2701-2022, 2022.
Herrington, T. C., Fletcher, C. G., and Kropp, H.: Validation of Pan-Arctic Soil Temperatures in Modern Reanalysis and Data Assimilation Systems, The Cryosphere, 18, 1835–1861, https://doi.org/10.5194/tc-18-1835-2024, 2024.
Hu, G., Zhao, L., Li, R., Wu, X., Wu, T., Xie, C., Zhu, X., and Su, Y.: Variations in soil temperature from 1980 to 2015 in permafrost regions on the Qinghai-Tibetan Plateau based on observed and reanalysis products, Geoderma, 337, 893–905, https://doi.org/10.1016/j.geoderma.2018.10.044, 2019.
Mortimer, C., Mudryk, L., Derksen, C., Luojus, K., Brown, R., Kelly, R., and Tedesco, M.: Evaluation of long-term Northern Hemisphere snow water equivalent products, The Cryosphere, 14, 1579–1594, https://doi.org/10.5194/tc-14-1579-2020, 2020.
Wang, C., Graham, R. M., Wang, K., Gerland, S., and Granskog, M. A.: Comparison of ERA5 and ERA-Interim near-surface air temperature, snowfall and precipitation over Arctic sea ice: effects on sea ice thermodynamics and evolution, The Cryosphere, 13, 1661–1679, https://doi.org/10.5194/tc-13-1661-2019, 2019.
Citation: https://doi.org/10.5194/egusphere-2025-575-RC1 -
AC1: 'Quick Reply on RC1', Bin Cao, 02 Apr 2025
We thank Anonymous Referee #1 for their constructive comments. While waiting for other comments on our preprint before replying in detail, we want to briefly comment now on two issues:
(1) Anonymous Referee #1: "I recommend that the authors extend their analysis to explicitly quantify what proportion of the uncertainties or spread in MAAT and SWE can be attributed to the low station density in cold regions, and what proportion can be attributed to the inadequate process representation in the products.”
Response: This is an important point that has also been raised by colleagues who commented on an earlier version of the manuscript. We have made a deliberate decision here: A detailed and conclusive analysis of the causes for the large spread in cold regions will likely be an involved process requiring a broad range of knowledge, skills, and perspectives that differ from ours, and that will take time to bring together in a research project. We made a choice to expose our finding as a Brief Communication quickly to motivate and accelerate this research.
(2) Anonymous Referee #1: "Why was NCEP2 investigated over NCEP CFSR/CFSv2, for example?”
Response: In selecting reanalyses to include, we have opted to not include CFSR/CFSv2 because it mixes two differing simulation and assimilation systems.
Citation: https://doi.org/10.5194/egusphere-2025-575-AC1 -
AC3: 'Reply on RC1', Bin Cao, 20 May 2025
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2025/egusphere-2025-575/egusphere-2025-575-AC3-supplement.pdf
-
AC1: 'Quick Reply on RC1', Bin Cao, 02 Apr 2025
-
RC2: 'Comment on egusphere-2025-575', Anonymous Referee #2, 19 Apr 2025
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2025/egusphere-2025-575/egusphere-2025-575-RC2-supplement.pdf
-
AC4: 'Reply on RC2', Bin Cao, 20 May 2025
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2025/egusphere-2025-575/egusphere-2025-575-AC4-supplement.pdf
-
AC4: 'Reply on RC2', Bin Cao, 20 May 2025
Status: closed
-
CC1: 'Comment on egusphere-2025-575', Steven Margulis, 28 Feb 2025
This is nice work, highlighting the issue of the high uncertainty of estimates of cold-land processes in reanalysis that are often used for making assessment of snow-derived water availability and how it may be changing. Some recent work that compared some of these global products to an observationally-constrained snow reanalysis dataset shed similar light and may be worth including in the Introduction for context.
Fang, Y., Y. Liu, D. Li, H. Sun, and S.A. Margulis, 2023. Spatiotemporal snow water storage uncertainty in the midlatitude American Cordillera, The Cryosphere, 17, 5175–5195, https://doi.org/10.5194/tc-17-5175-2023
Liu, Y., Y. Fang, D. Li, and S.A. Margulis, 2022. How well do global snow products characterize snow storage in High Mountain Asia? Geophysical Research Letters, 49, e2022GL100082. https://doi.org/10.1029/2022GL100082
Citation: https://doi.org/10.5194/egusphere-2025-575-CC1 -
AC2: 'Reply on CC1', Bin Cao, 20 May 2025
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2025/egusphere-2025-575/egusphere-2025-575-AC2-supplement.pdf
-
AC2: 'Reply on CC1', Bin Cao, 20 May 2025
-
RC1: 'Comment on egusphere-2025-575', Anonymous Referee #1, 01 Apr 2025
General Comments
Cao and Gruber investigate the performance of five modern reanalyses (JRA-3Q, ERA5, MERRA-2, JRA-55 and NCEP2) over cold regions, with a focus on air temperature and snow water equivalent (SWE). They show that the ensemble spread in mean annual air temperature (MAAT) in reanalyses is up 90% greater over cold regions (defined as regions with a MAAT <0oC) relative to regions with a MAAT ≥ 0oC. The study explicitly shows the relationship between station density and ensemble spread for both SWE and MAAT and is able to show that the reduced reanalysis performance is at least partially related to the low station density over cold regions.
The bulk of the conclusions come from Figure 1, which show the average ensemble spread in MAAT (Panel A) and SWE (Panel B) binned by MAAT. The station density in each MAAT bin, and the proportion of the grid cells within the MAAT bin covered by ice sheets and glaciers, snow cover, permafrost, and seasonally frozen ground is also shown. Herrington et al. (2024) showed a similar plot for soil temperature, that identified the average reanalysis spread in soil temperatures binned by MAAT, against sample size, though it didn’t explicitly consider station density, or the proportion of the grid cells covered by the cryosphere – so this is a novel analysis (along with the focus on SWE and MAAT).
While Figure 1 clearly shows a clear correlation between ensemble spread, station density, and the presence of cryospheric elements, there is no attempt to separate the contributions of low station density from those related to inadequate representation of cold region processes in reanalyses. As a reader it raises questions as to what the relative contributions of station density, and inadequate process representation to the ensemble spread in MAAT and SWE are?
To me, the novel science is in quantifying what proportion of the uncertainty can be attributed to station density, and what proportion is related to inadequate representation of cold region processes, which have been thoroughly discussed in the literature (e.g. Broxton et al., 2016; Cao et al., 2020, 2022; Hu et al., 2019; Mortimer et al., 2020; Wang et al., 2019). Thus, I recommend that the authors extend their analysis to explicitly quantify what proportion of the uncertainties or spread in MAAT and SWE can be attributed to the low station density in cold regions, and what proportion can be attributed to the inadequate process representation in the products. This will greatly enhance the contribution of the paper to the literature and provide the community with useful and quantifiable estimates of the uncertainty attribution.
Specific Comments
P2, L31: Why was NCEP2 investigated over NCEP CFSR/CFSv2, for example? NCEP CFSR/CFSv2 is available at a much higher resolution than NCEP2, and is available over the period of analysis (1991-2020).
P2, L36-L37: What do the authors mean by the statement “the decades 1991-2020 were used, likely a period of high quality for reanalyses.” Is there a particular standard by which the authors determined this? Some clarification would be helpful.
P2, L38: What is the CDS? It appears that CDS is in brackets, but the acronym is not defined. I presume this may be the Climate Data Store?
P4, L101: The all 5-reanalysis value for MAAT spread was (1.5oC, 0.5oC-3.0oC) - is this statistically different from the value for the 4DVar reanalyses reported here?
P10, Figure 1: What do the dashed lines represent in Figure 1? I don’t see a dashed line in the figure legend?
P11, Figure 2: It may be helpful to have a "difference" panel between the All 5 reanalyses and the three 4DVar reanalyses to highlight the regions with the largest differences, particularly for SWE, since it is a little harder to notice the differences in Panel B.
Technical Comments
P2, L38: replace “for” in “for International Earth Science Information Network…” with “from”.
P2, L81: replace “were” in the sentence “were slightly more permafrost…” with “where”.
References
Broxton, P. D., Zeng, X., and Dawson, N.: Why Do Global Reanalyses and Land Data Assimilation Products Underestimate Snow Water Equivalent?, J. Hydrometeorol., 17, 2743–2761, https://doi.org/10.1175/JHM-D-16-0056.1, 2016.
Cao, B., Gruber, S., Zheng, D., and Li, X.: The ERA5-Land soil temperature bias in permafrost regions, The Cryosphere, 14, 2581–2595, https://doi.org/10.5194/tc-14-2581-2020, 2020.
Cao, B., Arduini, G., and Zsoter, E.: Brief communication: Improving ERA5-Land soil temperature in permafrost regions using an optimized multi-layer snow scheme, The Cryosphere, 16, 2701–2708, https://doi.org/10.5194/tc-16-2701-2022, 2022.
Herrington, T. C., Fletcher, C. G., and Kropp, H.: Validation of Pan-Arctic Soil Temperatures in Modern Reanalysis and Data Assimilation Systems, The Cryosphere, 18, 1835–1861, https://doi.org/10.5194/tc-18-1835-2024, 2024.
Hu, G., Zhao, L., Li, R., Wu, X., Wu, T., Xie, C., Zhu, X., and Su, Y.: Variations in soil temperature from 1980 to 2015 in permafrost regions on the Qinghai-Tibetan Plateau based on observed and reanalysis products, Geoderma, 337, 893–905, https://doi.org/10.1016/j.geoderma.2018.10.044, 2019.
Mortimer, C., Mudryk, L., Derksen, C., Luojus, K., Brown, R., Kelly, R., and Tedesco, M.: Evaluation of long-term Northern Hemisphere snow water equivalent products, The Cryosphere, 14, 1579–1594, https://doi.org/10.5194/tc-14-1579-2020, 2020.
Wang, C., Graham, R. M., Wang, K., Gerland, S., and Granskog, M. A.: Comparison of ERA5 and ERA-Interim near-surface air temperature, snowfall and precipitation over Arctic sea ice: effects on sea ice thermodynamics and evolution, The Cryosphere, 13, 1661–1679, https://doi.org/10.5194/tc-13-1661-2019, 2019.
Citation: https://doi.org/10.5194/egusphere-2025-575-RC1 -
AC1: 'Quick Reply on RC1', Bin Cao, 02 Apr 2025
We thank Anonymous Referee #1 for their constructive comments. While waiting for other comments on our preprint before replying in detail, we want to briefly comment now on two issues:
(1) Anonymous Referee #1: "I recommend that the authors extend their analysis to explicitly quantify what proportion of the uncertainties or spread in MAAT and SWE can be attributed to the low station density in cold regions, and what proportion can be attributed to the inadequate process representation in the products.”
Response: This is an important point that has also been raised by colleagues who commented on an earlier version of the manuscript. We have made a deliberate decision here: A detailed and conclusive analysis of the causes for the large spread in cold regions will likely be an involved process requiring a broad range of knowledge, skills, and perspectives that differ from ours, and that will take time to bring together in a research project. We made a choice to expose our finding as a Brief Communication quickly to motivate and accelerate this research.
(2) Anonymous Referee #1: "Why was NCEP2 investigated over NCEP CFSR/CFSv2, for example?”
Response: In selecting reanalyses to include, we have opted to not include CFSR/CFSv2 because it mixes two differing simulation and assimilation systems.
Citation: https://doi.org/10.5194/egusphere-2025-575-AC1 -
AC3: 'Reply on RC1', Bin Cao, 20 May 2025
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2025/egusphere-2025-575/egusphere-2025-575-AC3-supplement.pdf
-
AC1: 'Quick Reply on RC1', Bin Cao, 02 Apr 2025
-
RC2: 'Comment on egusphere-2025-575', Anonymous Referee #2, 19 Apr 2025
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2025/egusphere-2025-575/egusphere-2025-575-RC2-supplement.pdf
-
AC4: 'Reply on RC2', Bin Cao, 20 May 2025
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2025/egusphere-2025-575/egusphere-2025-575-AC4-supplement.pdf
-
AC4: 'Reply on RC2', Bin Cao, 20 May 2025
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 778 | 67 | 34 | 879 | 35 | 30 | 41 |
- HTML: 778
- PDF: 67
- XML: 34
- Total: 879
- Supplement: 35
- BibTeX: 30
- EndNote: 41
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
This is nice work, highlighting the issue of the high uncertainty of estimates of cold-land processes in reanalysis that are often used for making assessment of snow-derived water availability and how it may be changing. Some recent work that compared some of these global products to an observationally-constrained snow reanalysis dataset shed similar light and may be worth including in the Introduction for context.
Fang, Y., Y. Liu, D. Li, H. Sun, and S.A. Margulis, 2023. Spatiotemporal snow water storage uncertainty in the midlatitude American Cordillera, The Cryosphere, 17, 5175–5195, https://doi.org/10.5194/tc-17-5175-2023
Liu, Y., Y. Fang, D. Li, and S.A. Margulis, 2022. How well do global snow products characterize snow storage in High Mountain Asia? Geophysical Research Letters, 49, e2022GL100082. https://doi.org/10.1029/2022GL100082