the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 06 Feb 2024
Performance evaluation of UKESM1 for surface ozone across the pan-tropics
Abstract. Surface ozone monitoring sites in the tropics are limited, despite the risk that surface ozone poses to human health, tropical forest, and crop productivity. Atmospheric chemistry models allow us to assess ozone exposure in unmonitored locations and evaluate the potential influence of changing policies and climate on air quality, human health, and ecosystem integrity. Here, we utilise in situ ozone measurements from ground-based stations in the pan-tropics to evaluate ozone from the UK Earth system model, UKESM1, with a focus on remote sites. The study includes ozone data from areas with limited previous data, notably Tropical South America, central Africa, and tropical North Australia. Evaluating UKESM1 against observations beginning in 1987 onwards, we show that UKESM1 is able to capture changes in surface ozone concentration at different temporal resolutions, albeit with a systematic high bias of 18.1 nmol mol-1 on average. We use the Diurnal Ozone Range (DOR) as a metric for evaluation and find that UKESM1 captures the observed DOR (mean bias of 2.7 nmol mol-1 and RMSE of 7.1 nmol mol-1) and the trend in DOR with location and season. Results from this study demonstrate the applicability of hourly output from UKESM1 for human and ecosystem health-based impact assessments, increase confidence in model projections, and highlight areas that would benefit from further observations. Indeed, hourly surface ozone data has been crucial to this study, and we encourage other modelling groups to include hourly surface ozone output as a default.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(2377 KB)
-
Supplement
(1826 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2377 KB) - Metadata XML
-
Supplement
(1826 KB) - BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2023-2937', Anonymous Referee #1, 19 Feb 2024
This manuscript describes an assessment of surface ozone in the Tropics from the UKESM1 model, demonstrating that mixing ratios are systematically high across all sites but that the average diurnal variation is reproduced well. It is a competent study, and scientifically sound, but it provides little fresh insight or understanding of either model performance or atmospheric behavior. A few simple sensitivity studies to investigate the cause of the biases would have made the paper substantially stronger, and this is a missed opportunity. It is highly likely that monthly mean averaging of biomass burning emissions is one source of error, as speculated, and that representation of the vertical profile in the lower boundary layer is another, but no solid evidence is provided. I feel that this needs to be addressed to at least some extent before the paper is suitable for publication.
General CommentsOne premise of the paper is that reasonable representation of the diurnal ozone range is a strength, despite a factor of two overestimation of ozone mixing ratios. Given that both deposition processes and titration by NO are first order in ozone, there is reason to expect proportional biases in DOR and ozone. Why should good representation of DOR give us confidence in the model, and what does this reveal about the source of the bias?
A case is made for more measurements of surface ozone across Tropical regions. While these would certainly be valuable, this need is not an outcome arising from this study, where it is clear that an otherwise well-tested model is unable to reproduce the surface ozone measurements we already have. A more relevant call would therefore be to investigate and diagnose these biases in models to fully explain and ideally eliminate it (and I believe that this paper should make the first steps towards doing this).
It is clear that surface ozone from global models such as UKESM1 cannot be used for reliable health and ecosystem impact assessments at the current time. The paper manages to arrive at the opposite conclusion (line 457). While bias correction can remove model errors effectively, it can also remove the need for a model in the first place (there are now a wealth of machine-learned observation-based surface ozone climatologies available). A much stronger argument for use of models is needed, otherwise the point made here needs to be substantially reframed in the paper.
Which of the stations considered are expected to be representative of the model grid used here (based on geospatial homogeneity of land cover and emissions) and which ones are not? These issues are touched on in section 4.2, but no explicit assessment is made, and this makes interpretation of biases more difficult.
For context, it is important to comment on how well UKESM1 represents surface ozone at midlatitudes and other regions outside the Tropics, and to provide a more critical assessment of how it compares with other global models in representing surface ozone (mentioned very briefly without detail on line 77). Reference to existing studies is fine for this, but the information is important for context, and the paper would be of wider interest if the issues are common to other models as well as just UKESM1.
Specific CommentsL.142: The 12 Tg/yr flux of soil NOx is as N, NO or NO2? If this flux is quantified, it would be useful to give the average or range of the lightning NOx source, and perhaps also the bVOC emissions.
L.147: This sentence belongs in the first paragraph of the section, before the discussion of emissions. It would also be helpful to indicate the level of complexity represented in the NMVOC chemistry (just longer-lived VOC and isoprene, or some treatment of other reactive VOC?)
L.163, 165: it is distracting to the reader to see references to Figures later in the results section while still in the methods; please remove these and present the analysis information in a more generic way here. Detail relevant to a specific figure should be included where that figure is presented.
L.205: clarification needed: does the standard deviation here quantify the interannual variability (or the seasonal variability)?
L.299: A p-value of this magnitude suggests incorrect application of statistical methods.
L.317: Subtitle better as "How well does UKESM1 reproduce...."
L.320: It is not clear what "pattern" refers to here (diurnal, seasonal, or spatial variation?)
L.337: model resolution is not a cause of model bias, it is a structural characteristic which leads to biases from the processes that are represented.
L.381: interactive natural precursor emissions probably are very important for reproducing tropical ozone, but your studies haven't shown this. A simple sensitivity study would allow you to confirm (and quantify) this effect.
L.453: resolution is not an attributable cause of bias, please rephrase here.
L.457: It is clear that surface ozone from global models such as UKESM1 should not be used for health and ecosystem impact assessments at the current time. Substantial revision of this point is required.
Fig S1 is cramped, although would be more readable if included in a vector format rather than as an image. The blue lines would be more accurately described as green (although the overlay on yellow bars makes this difficult to discern).
Fig S3: In light of the points about temperature sensitivity raised in section 2.3, it would be useful to include the mean temperatures and precipitation from UKESM1 in the table component to highlight how this compares with the reanalysis.
Fig S8: If site T1 does not have data between April and December then these points should not be joined on the graph.
Technical CorrectionsThere are some weaknesses in writing style in places that could be removed with a thorough proof-read. Other minor points:
Table 1: lat/lon of grid cell centers are not needed to four decimal places (one would be sufficient)
L.115: Table 1 indicates that Sao Paulo spans three gridcells, but the text states two; please correct text or table as appropriate.
L.156: ppb used here, but nmol/mol used elsewhere
L.192: add units: nmol/mol after 13.0. Note also that the percentage biases in the following sentence can't justify quotation to one decimal place.
L.213: trends conventionally refer to changes; rephrase here (also at L.266)
L.279: "factors rather than..." meaning of this sentence unclear, please rephrase
L.441: "across" not needed
L.459: "and is"
Figs 2, 3: x-axis interval in right panel should be 10 for consistency with y-axis.
Citation: https://doi.org/10.5194/egusphere-2023-2937-RC1 - AC2: 'Reply on RC2', Flossie Brown, 19 Jun 2024
-
RC2: 'Comment on egusphere-2023-2937', Anonymous Referee #2, 11 Mar 2024
In this manuscript, the authors evaluate surface ozone simulated by UKESM1 in the Tropics. The topic of the manuscript is scientifically very relevant for for the community but I wish the authors enhance their process understanding. They use a number of statistical methods to quantify the model performance. However, often the interpretation comes too short and thus all in all many aspects like the systematic bias of UKESM1 remain unexplained.
To compare station measurements with zhe model (grid cell) output the authors average the measurement data within one grid box. I think this is inaccurate and introduces uncertainty, since the stations have, as they mention later, different meteorological conditions. At urban stations, in particular, a complex chemistry at sub-grid scale occurs and thus they are hard to represent at coarse spatial resolution of ~140 km like here.
The authors apply UKESM1 in a free-running mode, as far as I understand. From my knowledge, this is very uncommon for a model evaluation/ comparison with measurements. Does the model represent the meteorology in the studied regions realistically?
Also, the usage of the diurnal ozone range for showing the model's feasability to capture the diurnal varaition of surface ozone, does not convince me.
Minor comments:
The description of VOC/NOx-limited regimes could be more clear (l. 63 ff)
l. 72: 'good test space' -> 'much study potentilal'
l. 145: Why don't you use the more recent estimates by Sinderalova et al. 2022?
What is the global annual lNOX emission?
Don't you think that enhanced process understanding and using a more complex SOA chemistry (for example) could resolve some of these model biases? Can you state whether the biases are model-specific and why? (l 385 ff.)
Citation: https://doi.org/10.5194/egusphere-2023-2937-RC2 - AC2: 'Reply on RC2', Flossie Brown, 19 Jun 2024
-
CC1: 'Comment on egusphere-2023-2937', Owen Cooper, 16 Mar 2024
This comment can be found in the attached pdf.
-
AC1: 'Reply on CC1', Flossie Brown, 19 Jun 2024
Thank you for taking the time to comment on our manuscript. We would primarily like to confirm that all data used in this study is open access (details included in manuscript). In particular, we would like to highlight new measurements at Daintree, Yangambi and Barro Colorado at DOI 10.5281/zenodo.10252770, which we hope will be useful to the TOAR Community. Further responses are attached.
-
AC1: 'Reply on CC1', Flossie Brown, 19 Jun 2024
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2023-2937', Anonymous Referee #1, 19 Feb 2024
This manuscript describes an assessment of surface ozone in the Tropics from the UKESM1 model, demonstrating that mixing ratios are systematically high across all sites but that the average diurnal variation is reproduced well. It is a competent study, and scientifically sound, but it provides little fresh insight or understanding of either model performance or atmospheric behavior. A few simple sensitivity studies to investigate the cause of the biases would have made the paper substantially stronger, and this is a missed opportunity. It is highly likely that monthly mean averaging of biomass burning emissions is one source of error, as speculated, and that representation of the vertical profile in the lower boundary layer is another, but no solid evidence is provided. I feel that this needs to be addressed to at least some extent before the paper is suitable for publication.
General CommentsOne premise of the paper is that reasonable representation of the diurnal ozone range is a strength, despite a factor of two overestimation of ozone mixing ratios. Given that both deposition processes and titration by NO are first order in ozone, there is reason to expect proportional biases in DOR and ozone. Why should good representation of DOR give us confidence in the model, and what does this reveal about the source of the bias?
A case is made for more measurements of surface ozone across Tropical regions. While these would certainly be valuable, this need is not an outcome arising from this study, where it is clear that an otherwise well-tested model is unable to reproduce the surface ozone measurements we already have. A more relevant call would therefore be to investigate and diagnose these biases in models to fully explain and ideally eliminate it (and I believe that this paper should make the first steps towards doing this).
It is clear that surface ozone from global models such as UKESM1 cannot be used for reliable health and ecosystem impact assessments at the current time. The paper manages to arrive at the opposite conclusion (line 457). While bias correction can remove model errors effectively, it can also remove the need for a model in the first place (there are now a wealth of machine-learned observation-based surface ozone climatologies available). A much stronger argument for use of models is needed, otherwise the point made here needs to be substantially reframed in the paper.
Which of the stations considered are expected to be representative of the model grid used here (based on geospatial homogeneity of land cover and emissions) and which ones are not? These issues are touched on in section 4.2, but no explicit assessment is made, and this makes interpretation of biases more difficult.
For context, it is important to comment on how well UKESM1 represents surface ozone at midlatitudes and other regions outside the Tropics, and to provide a more critical assessment of how it compares with other global models in representing surface ozone (mentioned very briefly without detail on line 77). Reference to existing studies is fine for this, but the information is important for context, and the paper would be of wider interest if the issues are common to other models as well as just UKESM1.
Specific CommentsL.142: The 12 Tg/yr flux of soil NOx is as N, NO or NO2? If this flux is quantified, it would be useful to give the average or range of the lightning NOx source, and perhaps also the bVOC emissions.
L.147: This sentence belongs in the first paragraph of the section, before the discussion of emissions. It would also be helpful to indicate the level of complexity represented in the NMVOC chemistry (just longer-lived VOC and isoprene, or some treatment of other reactive VOC?)
L.163, 165: it is distracting to the reader to see references to Figures later in the results section while still in the methods; please remove these and present the analysis information in a more generic way here. Detail relevant to a specific figure should be included where that figure is presented.
L.205: clarification needed: does the standard deviation here quantify the interannual variability (or the seasonal variability)?
L.299: A p-value of this magnitude suggests incorrect application of statistical methods.
L.317: Subtitle better as "How well does UKESM1 reproduce...."
L.320: It is not clear what "pattern" refers to here (diurnal, seasonal, or spatial variation?)
L.337: model resolution is not a cause of model bias, it is a structural characteristic which leads to biases from the processes that are represented.
L.381: interactive natural precursor emissions probably are very important for reproducing tropical ozone, but your studies haven't shown this. A simple sensitivity study would allow you to confirm (and quantify) this effect.
L.453: resolution is not an attributable cause of bias, please rephrase here.
L.457: It is clear that surface ozone from global models such as UKESM1 should not be used for health and ecosystem impact assessments at the current time. Substantial revision of this point is required.
Fig S1 is cramped, although would be more readable if included in a vector format rather than as an image. The blue lines would be more accurately described as green (although the overlay on yellow bars makes this difficult to discern).
Fig S3: In light of the points about temperature sensitivity raised in section 2.3, it would be useful to include the mean temperatures and precipitation from UKESM1 in the table component to highlight how this compares with the reanalysis.
Fig S8: If site T1 does not have data between April and December then these points should not be joined on the graph.
Technical CorrectionsThere are some weaknesses in writing style in places that could be removed with a thorough proof-read. Other minor points:
Table 1: lat/lon of grid cell centers are not needed to four decimal places (one would be sufficient)
L.115: Table 1 indicates that Sao Paulo spans three gridcells, but the text states two; please correct text or table as appropriate.
L.156: ppb used here, but nmol/mol used elsewhere
L.192: add units: nmol/mol after 13.0. Note also that the percentage biases in the following sentence can't justify quotation to one decimal place.
L.213: trends conventionally refer to changes; rephrase here (also at L.266)
L.279: "factors rather than..." meaning of this sentence unclear, please rephrase
L.441: "across" not needed
L.459: "and is"
Figs 2, 3: x-axis interval in right panel should be 10 for consistency with y-axis.
Citation: https://doi.org/10.5194/egusphere-2023-2937-RC1 - AC2: 'Reply on RC2', Flossie Brown, 19 Jun 2024
-
RC2: 'Comment on egusphere-2023-2937', Anonymous Referee #2, 11 Mar 2024
In this manuscript, the authors evaluate surface ozone simulated by UKESM1 in the Tropics. The topic of the manuscript is scientifically very relevant for for the community but I wish the authors enhance their process understanding. They use a number of statistical methods to quantify the model performance. However, often the interpretation comes too short and thus all in all many aspects like the systematic bias of UKESM1 remain unexplained.
To compare station measurements with zhe model (grid cell) output the authors average the measurement data within one grid box. I think this is inaccurate and introduces uncertainty, since the stations have, as they mention later, different meteorological conditions. At urban stations, in particular, a complex chemistry at sub-grid scale occurs and thus they are hard to represent at coarse spatial resolution of ~140 km like here.
The authors apply UKESM1 in a free-running mode, as far as I understand. From my knowledge, this is very uncommon for a model evaluation/ comparison with measurements. Does the model represent the meteorology in the studied regions realistically?
Also, the usage of the diurnal ozone range for showing the model's feasability to capture the diurnal varaition of surface ozone, does not convince me.
Minor comments:
The description of VOC/NOx-limited regimes could be more clear (l. 63 ff)
l. 72: 'good test space' -> 'much study potentilal'
l. 145: Why don't you use the more recent estimates by Sinderalova et al. 2022?
What is the global annual lNOX emission?
Don't you think that enhanced process understanding and using a more complex SOA chemistry (for example) could resolve some of these model biases? Can you state whether the biases are model-specific and why? (l 385 ff.)
Citation: https://doi.org/10.5194/egusphere-2023-2937-RC2 - AC2: 'Reply on RC2', Flossie Brown, 19 Jun 2024
-
CC1: 'Comment on egusphere-2023-2937', Owen Cooper, 16 Mar 2024
This comment can be found in the attached pdf.
-
AC1: 'Reply on CC1', Flossie Brown, 19 Jun 2024
Thank you for taking the time to comment on our manuscript. We would primarily like to confirm that all data used in this study is open access (details included in manuscript). In particular, we would like to highlight new measurements at Daintree, Yangambi and Barro Colorado at DOI 10.5281/zenodo.10252770, which we hope will be useful to the TOAR Community. Further responses are attached.
-
AC1: 'Reply on CC1', Flossie Brown, 19 Jun 2024
Peer review completion
Journal article(s) based on this preprint
Data sets
Empirical data for: Performance evaluation of UKESM1 for surface ozone across the pan-tropics Flossie Brown https://doi.org/10.5281/zenodo.10252770
Interactive computing environment
Python notebooks for: Performance evaluation of UKESM1 for surface ozone across the pan-tropics Flossie Brown https://github.com/flossie-brown/UKESM1_evaluation
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 464 | 171 | 33 | 668 | 62 | 21 | 19 |
- HTML: 464
- PDF: 171
- XML: 33
- Total: 668
- Supplement: 62
- BibTeX: 21
- EndNote: 19
Viewed (geographical distribution)
| Country | # | Views | % |
|---|---|---|---|
| United States of America | 1 | 258 | 38 |
| United Kingdom | 2 | 62 | 9 |
| China | 3 | 59 | 8 |
| India | 4 | 42 | 6 |
| Germany | 5 | 40 | 5 |
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
- 258
Gerd Folberth
Stephen Sitch
Paulo Artaxo
Marijn Bauters
Pascal Boeckx
Alexander W. Cheesman
Matteo Detto
Ninong Komala
Luciana Rizzo
Nestor Rojas
Ines dos Santos Vieira
Steven Turnock
Hans Verbeeck
Alfonso Zambrano
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2377 KB) - Metadata XML
-
Supplement
(1826 KB) - BibTeX
- EndNote
- Final revised paper