the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 16 Aug 2024
Validation of the version 4.5 MAESTRO ozone and NO2 measurements
Abstract. Launched aboard the Canadian satellite SCISAT in August 2003, the Measurement of Aerosol Extinction in the Stratosphere and Troposphere Retrieved by Occultation (MAESTRO) instrument has been measuring solar absorption spectra in the ultraviolet (UV) and visible part of the spectrum for more than 20 years. The UV channel measurements from MAESTRO are used to retrieve profiles of ozone from the short-wavelength end of the Chappuis band (UV-ozone) and NO2, while measurements made in the visible part of the spectrum are used to retrieve a separate ozone (Vis.-ozone) product. The latest ozone and NO2 profile products, version 4.5, have been released, which nominally cover the period from February 2004 to December 2023. Due to the buildup of an unknown contaminant, the UV-ozone and NO2 products are only viable up to June 2009 for NO2 and December 2009 for UV-ozone. This study presents comparisons of the version 4.5 MAESTRO ozone and NO2 measurements with coincident, both spatially and temporally, measurements from an ensemble of 11 other satellite limb-viewing instruments. In the stratosphere, the Vis.-ozone product was found to possess a small high bias, with stratosphere averaged relative differences between 2.3 % and 8.2 %, but overall good agreement with the comparison datasets is found. A similar bias, albeit with slightly poorer agreement, is found with the UV-ozone product in the stratosphere, with the average stratospheric agreement between MAESTRO and the other datasets ranging from 2.9 % to 11.9 %. For NO2, general agreement with the comparison datasets is only found in the range from 20 to 40 km. Within this range, MAESTRO is found to have a low bias for NO2, and most of the datasets agree to within 27.5 %, although the average agreement ranges from 8.5 % to 43.4 %.
- Preprint
(1048 KB) - Metadata XML
- BibTeX
- EndNote
Status: closed
-
RC1: 'Comment on egusphere-2024-2115', Robert Damadeo, 03 Sep 2024
Summary:
The authors present the results of the newest version (v4.5) of the MAESTRO O3 and NO2 data. Simple coincident event comparisons are made with 10 other platforms as well as the ACE-FTS instrument operating on the same spacecraft. The paper is well-written, the methodology of the comparisons is very straightforward, and the results are described in detail. This paper’s subject matter is well suited for this journal. My only concern is that the conclusions about some of the comparisons are likely misrepresenting the actual data quality because of the different sampling patterns of the instruments. I recommend this paper for publication after the following concerns are addressed.
Comments:
One simple omission, unless I missed it, is what is the end of the date range of data for instruments that are still operating used for this study.
Another clarification is regarding the event type. When the authors talk about MAESTRO SRs and SSs, does this refer to the spacecraft event type or the local event type? Obviously, given the nature of the analysis, it makes sense for the separation to be on the local event type and not the spacecraft event type. As far as I am aware, neither the MAESTRO nor the ACE-FTS data products specifically inform the user of the local event type, leaving it as a required calculation by the user. However, the orbit of SCISAT is somewhat unique and the local and spacecraft event types are very often not the same.
Lastly, the authors state that they separate coincidences based on the MAESTRO event type (SRs and SSs). However, when making coincident event comparisons with other solar occultation instruments, are the coincidences ensuring the same separation in the coincidences? If not, then there is likely too often a mixing of different kinds of airmasses (e.g., comparing a MAESTRO SR to a SAGE SS) and this analysis would need to be redone accordingly. My remaining comments are predicated upon the assumption that the stated comparisons show MAESTRO SRs/SSs compared to other solar occultation SRs/SSs respectively.
Because MAESTRO is a solar occultation instrument with sparse sampling associated with that technique and because the analyses performed here are based on coincidences between instruments, I believe that many of the comparisons are adversely affected by sampling biases created when analyzing these coincidences. This would be evident in the comparisons with other instruments that have their own sampling biases, creating comparisons that are not representative of the atmosphere as a whole and/or creating systematic differences in sampling locations. I have looked at this specifically for the SAGE instruments, but less noticeable sampling biases are also possible for GOMOS (stellar occultation) or Odin instruments (i.e., SMR and OSIRIS) that I recall having a hemispheric asymmetry in the overall sampling.
One indication of potential sampling biases is easily seen when looking at the mean SR/SS comparisons. For O3 throughout the lower and middle stratosphere, the impact of diurnal variability is minimal. If the events are generally evenly sampled in time and latitude, then the expectation is that the mean SR and mean SS profiles would overlap, as they do in the ACE-FTS comparison (naturally since every event is coincident) as well as with very dense samplers such as MLS, MIPAS, SCIAMACHY, and OMPS-LP. The fact that the mean SR and mean SS profiles just from the MAESTRO instrument begin to deviate from each other in the other comparisons is the first sign of potential sampling biases. The same is true for NO2 comparisons where, although diurnal variability is expected to be noticeable throughout the stratosphere, the scale of the diurnal variability just between the MAESTRO SR/SS profiles changes between different instrument comparisons.
For SAGE II, sampling biases created in coincident event comparisons are the most egregious. This is because not only is SAGE also a solar occultation instrument, but SAGE II was operating at a 50% duty cycle during the time of operational overlap with MAESTRO. I looked into the temporal and spatial distribution of coincident events (<8 hours, <1000 km) between SAGE II and ACE-FTS (which is data I had on-hand), assuming there would be almost identical sampling between ACE-FTS and MAESTRO, and found the following:
For SSs, all comparisons are basically confined to two small groupings: high southern latitudes (60-70) in late 2004 and high northern latitudes (60-70) in early 2005, noting that all of the southern latitude comparisons have SAGE II observations taking place at a systematically more northern latitude than MAESTRO observations. This systematic offset in latitude could create overall biases in the comparisons. In both cases, this means observations were taking place at a time and place of higher vortex variability, which would likely result in different standard deviations and reduced correlations.
For SRs, all comparisons are basically confined to another two groupings: high southern latitudes (50-70) in late 2004 and another semi-global patch (50N-40S) in early-to-mid 2005, noting that almost ALL of these comparisons have SAGE II observations taking place at a systematically more northern latitude than MAESTRO observations, in some cases exceeding a difference in latitude of 5 degrees. This systematic bias in spatial sampling likely contributes to the systematic bias in O3 seen in comparisons with SAGE II SRs.
For SAGE III/M3M, sampling biases created in coincident event comparisons are also somewhat unique because of the combination of a sunsync orbit with a solar occultation instrument. The effect of this is that all of SAGE III/M3M spacecraft SSs/SRs are observed in the northern/southern hemisphere. However, there is a distinction between spacecraft event type and local event type. For SAGE II, which was in a mid-inclination orbit, the two are almost always the same. For SAGE III/M3M, all of the spacecraft SRs are actually local SSs, and most of the spacecraft SSs are local SSs with the exception of polar winter where they are local SRs. This means that the distribution of coincidences with SAGE III/M3M SSs do not have much of a sampling bias, but a significant one exists for SRs. All coincidences between SAGE III/M3M and MAESTRO SRs occur within a small grouping at high northern latitudes (55-75) in early 2005, with all SAGE III/M3M observations taking place at a systematically more northern latitude than MAESTRO with a minimum offset of 5 degrees in latitude. Additionally, I compute a ratio of coincident SS events to SR events of nearly 10:1 (commensurate with the total number of local SSs versus SRs in the SAGE III/M3M dataset), which is very different from the 3:1 ratio the authors show. This makes me wonder if the authors really are not considering the different event types for comparison solar occultation instruments as I can get a similar number of coincidences if I ignore the SAGE III/M3M event type. If so, then this whole analysis really does need to be redone (at least for all SAGE instruments).
For SAGE III/ISS, there are again sampling biases from the combined orbital sampling of ISS and SCISAT. Strangely, all of the SR comparisons are in the northern hemisphere, while most of the SS comparisons are in the southern hemisphere, but the latitudinal extent gets broader as the years go by and start to expand into the southern hemisphere. While this could potentially be problematic if looking into drifts between the instruments, I don’t see any obvious source of bias in coincident event comparisons.
Citation: https://doi.org/10.5194/egusphere-2024-2115-RC1 -
AC1: 'Reply on RC1', Paul Jeffery, 28 Oct 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-2115/egusphere-2024-2115-AC1-supplement.pdf
-
AC1: 'Reply on RC1', Paul Jeffery, 28 Oct 2024
-
RC2: 'Comment on egusphere-2024-2115', Anonymous Referee #2, 03 Sep 2024
Summary and Significance
This paper presents a validation of version 4.5 O3 and NO2 from the MAESTRO instrument. Both UV and visible O3 products are considered. The MAESTRO observations are compared to observations from many other satellite limb instruments. In general, MAESTRO visible O3 agrees well with the other datasets between 20 and 50 km, while MAESTRO UV O3 has good agreement from 15-40 km. MAESTRO NO2 is biased low.
A thourough validation of the MAESTRO observations is important for anyone who wishes to use the data. The paper is well written and suitable for publication after some minor additions.
Questions and Comments
- Abstract: add a sentence discussing what has changed since the previous MAESTRO data version.
- Section 2.1.1: How were the values used for data filtering determined? Why filter using both vmr thresholds and a MAD filter?
- Line 179/180: The v7.2 OSIRIS retrieval does not include the MAD filter, or any manual inspection.
- Section 2.6.1: The SAGE II NO2 from sunrise occultations is affected by a thermal shock and so these data are considered a “research product” (Damadeo et al. 2013). This is probably worth mentioning since you are using the data. This could also be a reason that SAGE II sunrise NO2 is the only dataset with less NO2 than MAESTRO in figure 10.
- Line 481: Why look all the way down to 0 km? I don’t think these datasets are expected to be reliable in the troposphere.
- Section 3:
- Why didn’t you consider diurnal variations in O3? This becomes relevant in the upper stratosphere, above ~40 km (e.g. Strode et al. 2022). MAESTRO O3 might agree better with the non-solar occultation datasets at these higher altitudes if you include diurnal scaling. It would be good to apply the Strode et al. (2022) scaling factors to one of your O3 comparison datasets to see if this makes a difference.
- Is it reasonable to use scaling factor based on 2017-2022 to scale observations from 2004-2009? I know the Strode et al. (2022) paper says that the interannual variability is small enough that it can be neglected. But I feel like this is hard to claim based on only 4 years, and it does not actually look that small to me in their paper (fig. 6). I understand it is not feasible to calculate scaling factors for other years yourself, but you could test the sensitivity to some extent using the existing values. I suggest scaling one of the datasets with the max/min scaling factors based on the 4 years available and seeing how that affects the results. Possibly the effect will average out in the mean profiles.
- Section 4.2.3: It would be useful to include some discussion about possible reasons for the differences between the two MAESTRO ozone products.
- Figure 12: Some of the large differences at lower altitudes could be because diurnal variations along the instruments’ line of sight are neglected in many of the retrievals (e.g. Dube et al. 2021). This is especially a problem with measurements near the terminator and could be mentioned as an additional source of bias.
- Line 900: “slightly uncorrelated” is not very precise. It looks like they are not correlated.
- Conclusion: An additional paragraph the clearly states when/where the MAESTRO O3 and NO2 can confidently be used for scientific purposes would be useful.
Damadeo, R. P., Zawodny, J. M., Thomason, L. W., & Iyer, N. (2013). SAGE version 7.0 algorithm: application to SAGE II. Atmospheric Measurement Techniques, 6(12), 3539-3561.
Dubé, K., Bourassa, A., Zawada, D., Degenstein, D., Damadeo, R., Flittner, D., & Randel, W. (2021). Accounting for the photochemical variation in stratospheric NO 2 in the SAGE III/ISS solar occultation retrieval. Atmospheric Measurement Techniques, 14(1), 557-566.
Strode, S. A., Taha, G., Oman, L. D., Damadeo, R., Flittner, D., Schoeberl, M., ... & Stauffer, R. (2022). SAGE III/ISS ozone and NO2 validation using diurnal scaling factors. Atmospheric Measurement Techniques, 15(20), 6145-6161.
Citation: https://doi.org/10.5194/egusphere-2024-2115-RC2 -
AC1: 'Reply on RC1', Paul Jeffery, 28 Oct 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-2115/egusphere-2024-2115-AC1-supplement.pdf
Status: closed
-
RC1: 'Comment on egusphere-2024-2115', Robert Damadeo, 03 Sep 2024
Summary:
The authors present the results of the newest version (v4.5) of the MAESTRO O3 and NO2 data. Simple coincident event comparisons are made with 10 other platforms as well as the ACE-FTS instrument operating on the same spacecraft. The paper is well-written, the methodology of the comparisons is very straightforward, and the results are described in detail. This paper’s subject matter is well suited for this journal. My only concern is that the conclusions about some of the comparisons are likely misrepresenting the actual data quality because of the different sampling patterns of the instruments. I recommend this paper for publication after the following concerns are addressed.
Comments:
One simple omission, unless I missed it, is what is the end of the date range of data for instruments that are still operating used for this study.
Another clarification is regarding the event type. When the authors talk about MAESTRO SRs and SSs, does this refer to the spacecraft event type or the local event type? Obviously, given the nature of the analysis, it makes sense for the separation to be on the local event type and not the spacecraft event type. As far as I am aware, neither the MAESTRO nor the ACE-FTS data products specifically inform the user of the local event type, leaving it as a required calculation by the user. However, the orbit of SCISAT is somewhat unique and the local and spacecraft event types are very often not the same.
Lastly, the authors state that they separate coincidences based on the MAESTRO event type (SRs and SSs). However, when making coincident event comparisons with other solar occultation instruments, are the coincidences ensuring the same separation in the coincidences? If not, then there is likely too often a mixing of different kinds of airmasses (e.g., comparing a MAESTRO SR to a SAGE SS) and this analysis would need to be redone accordingly. My remaining comments are predicated upon the assumption that the stated comparisons show MAESTRO SRs/SSs compared to other solar occultation SRs/SSs respectively.
Because MAESTRO is a solar occultation instrument with sparse sampling associated with that technique and because the analyses performed here are based on coincidences between instruments, I believe that many of the comparisons are adversely affected by sampling biases created when analyzing these coincidences. This would be evident in the comparisons with other instruments that have their own sampling biases, creating comparisons that are not representative of the atmosphere as a whole and/or creating systematic differences in sampling locations. I have looked at this specifically for the SAGE instruments, but less noticeable sampling biases are also possible for GOMOS (stellar occultation) or Odin instruments (i.e., SMR and OSIRIS) that I recall having a hemispheric asymmetry in the overall sampling.
One indication of potential sampling biases is easily seen when looking at the mean SR/SS comparisons. For O3 throughout the lower and middle stratosphere, the impact of diurnal variability is minimal. If the events are generally evenly sampled in time and latitude, then the expectation is that the mean SR and mean SS profiles would overlap, as they do in the ACE-FTS comparison (naturally since every event is coincident) as well as with very dense samplers such as MLS, MIPAS, SCIAMACHY, and OMPS-LP. The fact that the mean SR and mean SS profiles just from the MAESTRO instrument begin to deviate from each other in the other comparisons is the first sign of potential sampling biases. The same is true for NO2 comparisons where, although diurnal variability is expected to be noticeable throughout the stratosphere, the scale of the diurnal variability just between the MAESTRO SR/SS profiles changes between different instrument comparisons.
For SAGE II, sampling biases created in coincident event comparisons are the most egregious. This is because not only is SAGE also a solar occultation instrument, but SAGE II was operating at a 50% duty cycle during the time of operational overlap with MAESTRO. I looked into the temporal and spatial distribution of coincident events (<8 hours, <1000 km) between SAGE II and ACE-FTS (which is data I had on-hand), assuming there would be almost identical sampling between ACE-FTS and MAESTRO, and found the following:
For SSs, all comparisons are basically confined to two small groupings: high southern latitudes (60-70) in late 2004 and high northern latitudes (60-70) in early 2005, noting that all of the southern latitude comparisons have SAGE II observations taking place at a systematically more northern latitude than MAESTRO observations. This systematic offset in latitude could create overall biases in the comparisons. In both cases, this means observations were taking place at a time and place of higher vortex variability, which would likely result in different standard deviations and reduced correlations.
For SRs, all comparisons are basically confined to another two groupings: high southern latitudes (50-70) in late 2004 and another semi-global patch (50N-40S) in early-to-mid 2005, noting that almost ALL of these comparisons have SAGE II observations taking place at a systematically more northern latitude than MAESTRO observations, in some cases exceeding a difference in latitude of 5 degrees. This systematic bias in spatial sampling likely contributes to the systematic bias in O3 seen in comparisons with SAGE II SRs.
For SAGE III/M3M, sampling biases created in coincident event comparisons are also somewhat unique because of the combination of a sunsync orbit with a solar occultation instrument. The effect of this is that all of SAGE III/M3M spacecraft SSs/SRs are observed in the northern/southern hemisphere. However, there is a distinction between spacecraft event type and local event type. For SAGE II, which was in a mid-inclination orbit, the two are almost always the same. For SAGE III/M3M, all of the spacecraft SRs are actually local SSs, and most of the spacecraft SSs are local SSs with the exception of polar winter where they are local SRs. This means that the distribution of coincidences with SAGE III/M3M SSs do not have much of a sampling bias, but a significant one exists for SRs. All coincidences between SAGE III/M3M and MAESTRO SRs occur within a small grouping at high northern latitudes (55-75) in early 2005, with all SAGE III/M3M observations taking place at a systematically more northern latitude than MAESTRO with a minimum offset of 5 degrees in latitude. Additionally, I compute a ratio of coincident SS events to SR events of nearly 10:1 (commensurate with the total number of local SSs versus SRs in the SAGE III/M3M dataset), which is very different from the 3:1 ratio the authors show. This makes me wonder if the authors really are not considering the different event types for comparison solar occultation instruments as I can get a similar number of coincidences if I ignore the SAGE III/M3M event type. If so, then this whole analysis really does need to be redone (at least for all SAGE instruments).
For SAGE III/ISS, there are again sampling biases from the combined orbital sampling of ISS and SCISAT. Strangely, all of the SR comparisons are in the northern hemisphere, while most of the SS comparisons are in the southern hemisphere, but the latitudinal extent gets broader as the years go by and start to expand into the southern hemisphere. While this could potentially be problematic if looking into drifts between the instruments, I don’t see any obvious source of bias in coincident event comparisons.
Citation: https://doi.org/10.5194/egusphere-2024-2115-RC1 -
AC1: 'Reply on RC1', Paul Jeffery, 28 Oct 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-2115/egusphere-2024-2115-AC1-supplement.pdf
-
AC1: 'Reply on RC1', Paul Jeffery, 28 Oct 2024
-
RC2: 'Comment on egusphere-2024-2115', Anonymous Referee #2, 03 Sep 2024
Summary and Significance
This paper presents a validation of version 4.5 O3 and NO2 from the MAESTRO instrument. Both UV and visible O3 products are considered. The MAESTRO observations are compared to observations from many other satellite limb instruments. In general, MAESTRO visible O3 agrees well with the other datasets between 20 and 50 km, while MAESTRO UV O3 has good agreement from 15-40 km. MAESTRO NO2 is biased low.
A thourough validation of the MAESTRO observations is important for anyone who wishes to use the data. The paper is well written and suitable for publication after some minor additions.
Questions and Comments
- Abstract: add a sentence discussing what has changed since the previous MAESTRO data version.
- Section 2.1.1: How were the values used for data filtering determined? Why filter using both vmr thresholds and a MAD filter?
- Line 179/180: The v7.2 OSIRIS retrieval does not include the MAD filter, or any manual inspection.
- Section 2.6.1: The SAGE II NO2 from sunrise occultations is affected by a thermal shock and so these data are considered a “research product” (Damadeo et al. 2013). This is probably worth mentioning since you are using the data. This could also be a reason that SAGE II sunrise NO2 is the only dataset with less NO2 than MAESTRO in figure 10.
- Line 481: Why look all the way down to 0 km? I don’t think these datasets are expected to be reliable in the troposphere.
- Section 3:
- Why didn’t you consider diurnal variations in O3? This becomes relevant in the upper stratosphere, above ~40 km (e.g. Strode et al. 2022). MAESTRO O3 might agree better with the non-solar occultation datasets at these higher altitudes if you include diurnal scaling. It would be good to apply the Strode et al. (2022) scaling factors to one of your O3 comparison datasets to see if this makes a difference.
- Is it reasonable to use scaling factor based on 2017-2022 to scale observations from 2004-2009? I know the Strode et al. (2022) paper says that the interannual variability is small enough that it can be neglected. But I feel like this is hard to claim based on only 4 years, and it does not actually look that small to me in their paper (fig. 6). I understand it is not feasible to calculate scaling factors for other years yourself, but you could test the sensitivity to some extent using the existing values. I suggest scaling one of the datasets with the max/min scaling factors based on the 4 years available and seeing how that affects the results. Possibly the effect will average out in the mean profiles.
- Section 4.2.3: It would be useful to include some discussion about possible reasons for the differences between the two MAESTRO ozone products.
- Figure 12: Some of the large differences at lower altitudes could be because diurnal variations along the instruments’ line of sight are neglected in many of the retrievals (e.g. Dube et al. 2021). This is especially a problem with measurements near the terminator and could be mentioned as an additional source of bias.
- Line 900: “slightly uncorrelated” is not very precise. It looks like they are not correlated.
- Conclusion: An additional paragraph the clearly states when/where the MAESTRO O3 and NO2 can confidently be used for scientific purposes would be useful.
Damadeo, R. P., Zawodny, J. M., Thomason, L. W., & Iyer, N. (2013). SAGE version 7.0 algorithm: application to SAGE II. Atmospheric Measurement Techniques, 6(12), 3539-3561.
Dubé, K., Bourassa, A., Zawada, D., Degenstein, D., Damadeo, R., Flittner, D., & Randel, W. (2021). Accounting for the photochemical variation in stratospheric NO 2 in the SAGE III/ISS solar occultation retrieval. Atmospheric Measurement Techniques, 14(1), 557-566.
Strode, S. A., Taha, G., Oman, L. D., Damadeo, R., Flittner, D., Schoeberl, M., ... & Stauffer, R. (2022). SAGE III/ISS ozone and NO2 validation using diurnal scaling factors. Atmospheric Measurement Techniques, 15(20), 6145-6161.
Citation: https://doi.org/10.5194/egusphere-2024-2115-RC2 -
AC1: 'Reply on RC1', Paul Jeffery, 28 Oct 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-2115/egusphere-2024-2115-AC1-supplement.pdf
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 200 | 62 | 10 | 272 | 3 | 3 |
- HTML: 200
- PDF: 62
- XML: 10
- Total: 272
- BibTeX: 3
- EndNote: 3
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1