the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Mitigation of satellite OCO-2 CO2 biases in the vicinity of clouds with 3D calculations using the Education and Research 3D Radiative Transfer Toolbox (EaR3T)
Abstract. Accurate and continuous measurements of atmospheric carbon dioxide (CO2) are essential for climate change research and monitoring of emission reduction efforts. NASA's Orbiting Carbon Observatory (OCO-2/3) satellites have been deployed to measure the column-averaged CO2 dry air mixing ratio (XCO2) with very high precision. Although cloudy measurements are screened out, nearby clouds can still cause retrieval biases because the forward one-dimensional (1D) radiative transfer (RT) model used in the OCO retrieval algorithm does not account for the scattering induced by clouds in the vicinity of the OCO-2/3 footprints. These biases, referred to as the three-dimensional (3D) effects, can be quantified effectively using 3D-RT calculations, but these are computationally expensive, especially for hyperspectral applications (e.g., OCO-2/3). To reduce the prohibitive computational demands of 3D-RT radiance simulations across all three OCO spectral bands, this paper employs a linear approximation with two metrics (called slope and intercept) for each of the OCO bands that represent the 3D-RT perturbations on the OCO-2 spectra and accelerate the radiative transfer by a factor of 100. This is implemented by the Education and Research 3D Radiation Transfer Toolbox for OCO (EaR3T-OCO). EaR3T-OCO estimates OCO-2 satellite radiances using all available footprint-level data and imagery from the Aqua satellite, which orbits in close proximity to the OCO-2 satellite. EaR3T-OCO can calculate 3D-RT spectral perturbations for any OCO-2 footprint. These calculations can be used to spectrally adjust the OCO-2 radiance measurements with scene-dependent EaR3T-OCO perturbation calculations prior to the actual retrieval to undo cloud vicinity effects in the radiance spectra, which can subsequently be processed with the standard OCO-2 retrieval code. We find that this adjustment largely mitigates XCO2 retrieval biases in proximity to clouds over land – the first physics-based correction of 3D-RT effects on OCO-2/3 retrievals. Although the accelerated 3D-RT radiance adjustment step is faster than full 3D-RT calculations for all OCO spectral bands, it still requires at least as much computational effort as the XCO2 retrieval itself. To bypass 3D-RT altogether, the slope and intercept metrics are parameterized as a function of the weighted cloud distance of a footprint and several other scene parameters, all of which can be derived directly from Aqua-MODIS imagery. While this method is fastest and thus feasible for operational use, it requires careful validation for various surface and atmospheric conditions. For the case we analyzed, both the 3D-RT calculation method and the parametric bypass method successfully corrected XCO2 biases, which exceeded 2 ppm at the footprint level, and reached up to 0.7 ppm in the regional average. We find that the biases depend most strongly on the cloud field morphology and surface reflectance, but also on secondary factors such as aerosol layers and sun-sensor geometry.
- Preprint
(8310 KB) - Metadata XML
- BibTeX
- EndNote
Status: final response (author comments only)
-
CC1: 'Comment on egusphere-2024-1936', Jesse Loveridge, 28 Aug 2024
-
AC1: 'Reply on CC1', Yu-Wen Chen, 14 Nov 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-1936/egusphere-2024-1936-AC1-supplement.pdf
-
AC1: 'Reply on CC1', Yu-Wen Chen, 14 Nov 2024
-
RC1: 'Comment on egusphere-2024-1936', Anonymous Referee #1, 29 Aug 2024
General comments:
The paper presents a step forward to mitigate biases of retrieved CO2 concentrations from satellite observations (OCO-2/3) due to scattering of radiation by clouds in clear regions. 3D scattering effects are approximated by a linear fit of the radiances against selected wavelengths of a given spectral window. Based on another ongoing work, the authors claim that 3D scattering effects for a specific scene can be described by the slope and the intercept of the fit. These two parameters can be used to adjust the cloud contaminated observations to corresponding clear sky spectra. In a following step an exponential fit of the slope and the intercept against the effective distance to clouds is performed so that the adjustment of the spectra can be done without 3D radiative transfer (3D RT) simulations for each pixel. In order to avoid 3D RT calculations completely the so-called bypass method is also presented, which uses only observational data to obtain the slope and the intercept as a function of distance from the cloud and "other scene parameters" (the second is not shown in the paper).
The basic method using the 3DRT simulations requires cloud input data (cloud optical thickness, effective radius and cloud top height from MODIS observations), surface albedo (also from MODIS) and several assumptions (which are not fully described). It is rather difficult to obtain the realistic setup and therefore not straightforward to obtain the slope and intercept parameters.
The bypass method, which derives the parameters only from the observed radiances would be more practicable and it does not rely on other retrieval algorithms and assumptions on input to RT simulations, therefore this is in my opinion a promising approach.The methods are demonstrated for a particular scene, showing the retrieved CO2 concentration of the original OCO retrieval versus the adapted retrieval. For this particular case I see that the adapted retrieval gives smaller CO2 concentrations (up to 6ppm) in the cloud shadow (for which the correction is not designed for as mentioned in the text). For in-scattering, there is almost no difference as far as I can see in Fig.9. The corrected results are similar for both methods, the baseline and the bypass approaches. The truth is not known, and the authors therefore can not prove that they achieve an improvement. A way to validate the approach would be to use synthetic observations with known input concentration.
Certainly it is important to correct cloud effects in trace gas retrievals, therefore I think that the topic of the paper is appropriate for AMT. However, more validation is needed and I suggest to revise the paper in this respect. Further, several clarifications regarding the setup of the methods are required, for example it is not explained how cloud shadows are excluded.
Specific comments:- The abstract is relatively long, it could be focused a more on the two mitigation strategies and technical details about EaR3T-OCO could be shortened
- l.24 "These biases, referred to as the three-dimensional (3D) effects, can be quantified effectively using 3D-RT calculations, but these are computationally expensive, especially for hyperspectral applications (e.g., OCO-2/3)."
The authors refer later to the ALIS method for spectral Monte Carlo simulations. It is true that 3D RT is generally expensive but with ALIS spectral simulations are almost as fast as monochromatic simulations (see ALIS, Emde et al., 2011)
- l.44 "For the case we analyzed, both the 3D-RT calculation method and the
parametric bypass method successfully corrected XCO2 biases, which exceeded 2 ppm at the footprint level, and reached up to 0.7 ppm in the regional average."How do you know the the correction is successful, you get a difference but you do not know the true CO2 concentration?
An approach to validate the retrieval is to use synthetic data, as shown in the cited work by Kylling et al. 2022 for TROPOMI. A more systematic study on the mitigation of cloud scattering for TROPOMI is shown in Hu et al. 2022, who use 2D RT simulations to derive fits to correct retrieved airmass factors and validate this approach using realistic synthetic data based on Large Eddy simulations. The synthetic data (Emde et al. 2022) they are using includes also O2A band spectra and could possibly also be used to validate the mitigation approach for CO2 retrievals.- l. 46 "We find that the biases depend most strongly on the cloud field morphology and surface reflectance, but also on secondary factors such as aerosol ayers and sun-sensor geometry."
I assume with cloud field morphology, you mean the weighted distance to the clouds. The paper does not show how the bias depends on surface reflectance and on sun-sensor geometry. The impact of cloud scattering will certainly increase with increasing solar zenith angle but also for slant viewing angles (here only nadir view is shown). Also the cloud geometrical thickness is probably important.
- Eq.1: Remove 100%, because 100%=1. Or multiply by 100 and say that the unit of the perturbation is in per cent.
- Fig.1: Is the fitted line obtained by fitting the blue dots or the grey dots? You should show that both fits result in the same slope and intercept (you could include the two fitting lines and the corresponding equations).
- l. 146: "The intercept is related to the often-reported increase of reflectance near clouds, or decrease in shadows, whereas the slope accounts for spectroscopic effects."
This interpretation is correct as long as the spectral dependence of scattering can be neglected which is true for small spectral windows.
- Fig.2: It looks as if the main differences between the retrievals are in the cloud shadow region. Could you also show the image without the CO2 concentration included to see whether there is a cloud shadow at the place with higher CO2 concentrations?
- l. 267: "To determine the cloud optical thickness (COT) of each pixel, we run the RT model over several COT and derive the COT-radiance relationship by ourselves to ensure the radiance consistency in 1D-RT simulation."
This is not so clear. It means you do not use the cloud optical thickness as from the MODIS retrieval algorithm but an adjusted optical thickness that is needed as input for the 1D RT simulation to be consistent with the observed radiance? Isn't this exactly the same as the MODIS retrieval?
The retrieved optical thickness is of course biased by 3D effects because generally the reflectance is smaller in 3D simulations compared to 1D for the same vertical optical thickness due to photon leakage on the cloud sides. That means that the COT is underestimated, but this is not what you mean here?
- l.281: "MCARaTS iteratively traces the path of each photon and calculates the distribution of photons based on the final probability."
What do you mean with "distribution of photons" and "final probability"?
- l. 287: "The mean radiance and the standard deviation are then calculated from three runs to estimate the uncertainty."
Three samples are not sufficient to estimate the standard deviation. Why not running more simulations with less photons to get a better estimate?
- Eq. 4: Remove 100% which is equal to 1.
- l. 328: "Fig. 3a-b presents the 3D-RT simulation and MODIS observation of 650 nm using the COT, CER, and CTH shown in Fig. A3."
It is not fully clear how the cloud input is created from the MODIS data.
Why is CER seems constant (looks like this in Fig. A3)? Please provide the value of CER. How is the cloud vertically constructed? What is the cloud base height? What is the sun-observer geometry?
Can the method also be applied for ice clouds? Is it valid over ocean?How is the spectral albedo generated from MODIS data? A dataset and a method to generate hyperspectral surface albedo data from MODIS data is presented in Roccetti et al. 2024, could this be included in your model?
- l. 330: "The heat map in Fig. 3c shows a good agreement between the simulation and observation. As a result, we are confident that the simulation at other wavelengths is able to approach the actual condition."
More tests needed to draw this conclusion. How do other bands compare? At least one image in NIR should be shown.
Most of the points in the image correspond to clear sky. How is the correlation for the cloudy pixels only? This would show better whether the cloud input is realistic.
I think "scatter plot" is a better name than "heat map". Could you also include a colorbar?
- l. 345: "Employing a reduced number of wavelengths, uniformly distributed across the reflectance space, effectively minimizes computational demands while still permitting the derivation of 𝒔 and 𝒊 for the linear relationships within each band."
How are the wavelengths selected? Do you use the same set of wavelengths for all scenes?
Have you tested whether this relationship remains linear in different cloud situations? Clouds can shield the lower atmosphere so that due to the presence of clouds the amount of CO2 absorption is significantly decreased. I would expect non-linearity effects and would like to understand better why this relation should always be linear.
- l. 365: 1km² -> 1.25km²?
- l. 366: "We excluded the data if the 25 nearest grid points contained cloud pixels used in the RT simulation."
Does this mean you consider only completely clear sky pixels? What about partially cloudy pixels for which CO2 retrievals are also performed, if the cloud fraction is not too high?
- l. 381: "Though it is instructive to discuss both cloud brightening and cloud shadowing effects, Massie et al. (2023) determined that there are relatively few cloud shadow retrievals in the OCO-2 Lite files, since many observations impacted by shadowing are screened by the OCO-2 pre-retrieval cloud screening algorithms. Thereafter, bright area analyses are the primary focus of our study."
As mentioned before, in the specific scene that you present it seems that you obtain largest differences in a cloud shadow region?
- l. 387: "we identified an exponential decay relationship between the 3D cloud effect parameters and the effective horizontal cloud distance (𝐷, Fig. 6)"
What is the "bright area" (mentioned in the caption of Fig. 6) in the images, how do you select which is bright area and which is shadow?
I assume that this exponential decay is only valid in the bright area, because in the shadow there are abrubt changes in reflectance where the shadow ends. Is this the reason, why you say that your method is focused on the bright area?
In Fig. 6 I don't see the "background shading". Could you include colors corresponding to the density of the dots as in Fig.3?- Table 1: These parameters are valid only for the specific scene, correct? This should be clarified. Is the number of digits meaningful?
- l. 516: "However, the bypass method is less precise than conducting a 3D-RT simulation with our baseline approach to derive s and i on a pixel-by-pixel basis."
Why is it less precise? I think it could even be more precise because it does not rely on assumptions to produce the input for the 3DRT simulations.
- l. 517: "This bypass approach also disregards the presence of cloud shadows."
Why can the bypass approach not account for the shadows? Shadows are included in the observed radiances.
- Fig. 12: Are the results shown here based on the basic approach or the bypass approach? There are many shadows in the images. Do you include a cloud mask to exclude the shadows or how are they treated?
- l. 647: "... it allows the parameterization of the six spectral perturbation parameters themselves as a function of macroscopic scene parameters ..."
Which are the macroscopic scene parameters?
Couldn't one use only the spectral radiance observations to obtain the perturbation parameters using the bypass method?- l. 656: "While the bypass method does capture the significant modulators of the 3D cloud effects, including surface reflectance and sun-sensor geometry, it is not granular enough to consider detailed scene variables such as cloud top height, cloud morphology, or aerosol load."
Shouldn't the bypass method, since it uses reflectances containing all 3D cloud effects, capture all modulators?
References:Roccetti, G., Bugliaro, L., Gödde, F., Emde, C., Hamann, U., Manev, M., Sterzik, M. F., and Wehrum, C.: Development of a HAMSTER: Hyperspectral Albedo Maps dataset with high Spatial and TEmporal Resolution, EGUsphere [preprint], https://doi.org/10.5194/egusphere-2024-167, 2024.
Emde, C., Yu, H., Kylling, A., van Roozendael, M., Stebel, K., Veihelmann, B., and Mayer, B.: Impact of 3D cloud structures on the atmospheric trace gas products from UV–Vis sounders – Part 1: Synthetic dataset for validation of trace gas retrieval algorithms, Atmos. Meas. Tech., 15, 1587–1608, https://doi.org/10.5194/amt-15-1587-2022, 2022.
Yu, H., Emde, C., Kylling, A., Veihelmann, B., Mayer, B., Stebel, K., and Van Roozendael, M.: Impact of 3D cloud structures on the atmospheric trace gas products from UV–Vis sounders – Part 2: Impact on NO2 retrieval and mitigation strategies, Atmos. Meas. Tech., 15, 5743–5768, https://doi.org/10.5194/amt-15-5743-2022, 2022.
Citation: https://doi.org/10.5194/egusphere-2024-1936-RC1 -
AC2: 'Reply on RC1', Yu-Wen Chen, 14 Nov 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-1936/egusphere-2024-1936-AC2-supplement.pdf
-
AC2: 'Reply on RC1', Yu-Wen Chen, 14 Nov 2024
-
RC2: 'Comment on egusphere-2024-1936', Anonymous Referee #2, 16 Sep 2024
Overall, the subject of the study is very compelling and a significant contribution to the community. Especially considering future upcoming green house gas missions.
Major Comments
However, the study misses depth in how far the 3D cloud bias correction has been investigated. Major concern is that the developed approach has been applied only to a single, hand-picked scene. This is simply not enough to make any guesses towards the performance of the approach when applied operationally. The study states that it developed a software tool for the automated calculation of spectral radiances from OCO-2. However, the automation is not exploited to analyze a representative sample size of OCO-2 observations. Furthermore, for this single selected scene the strongest biases seem to be collocated with cloud shadows while the authors argue that those shadows are outside the scope of this study. For this research to be useful to the community it needs to show that it can be generalized (e.g. various SZA, ocean (where 3D cloud effects are strongest), land surface types, different cloud types, different viewing modes (nadir and glint)).
The study currently reads more like a description of the work that was performed rather than being focused on the outcome of the work. The outcome is what your reader is interested in. I would suggest picking either the 3D cloud correction based on the 3RT simulations or the ‘bypass’ method as the outcome of this work and explore the chosen method further (explore more scenes to better estimate performance once applied operationally).
Minor Comments
The paper often refers to qualitative statements that should be quantified or omitted. I pointed out individual instances below.
The abstract should be shortened and more to the point. What are the key takeaways from this study. Not necessary to expose all the ‘sausage making’ in the abstract.
I would suggest to merge section 1 and 2.
Specific comments by Line:
L20: quantify ‘high precision’ or omit
L20 – L23: Sentence starting with ‘Although …’ is hard to digest and should be simplified, maybe broken up.
L27: remove ‘with two metrics (called slope and intercept)’
L28: remove ‘and accelerate the radiative transfer by a factor of 100’
L31 – L35: Sentence starting in ‘EaRT-OCO .. ‘ -> move out of abstract.
L36: remove ‘– the first physics-based correction of 3D-RT effects on OCO-2/3 retrievals’
L37-L43: shorten, simplify discussion of ‘bypass’ method.
L62: quantify ‘accuracy’ requirement from the two cited studies.
L76: remove sentence ‘The cloud-related …’
L90-91: Restate comment that no ‘practical strategies’ have been developed to correct 3D cloud effects based on the physical understanding. The study by Mauceri et al (2023) uses physics derived variables to correct for 3D cloud biases.
L93: Please also include/cite work by Massie et al where they worked on correcting 3D cloud biases with linear fits to physics derived variables.
L106: ‘on the a footprint-by-footprint ‘
L126: specify that range ‘dynamic range of interest for reflectance’
L142-145: Hard to follow ‘Increased photon …’ . Please rewrite, expand.
L154: Why not use B11?
L244: ‘To mitigate excessive computational demands, we opt to use solely the wavelengths of the first footprint.’ -> how does this impact the results?
L262: how did the various reflectance thresholds influence the results.
L263: why did you need to develop a new cloud detection approach?
L298: ‘uncertainties’ : keep in mind that the uncertainties in s, I, depend on many more factors than captured by the uncertainty in the line fit. Thus, you would underestimate the true uncertainties with that approach.
L307: what are those ‘various processes’?
L310: The code on Gituhub is not the code used for the operational retrieval.
L320: explain terms in equation
L325: ‘parameters that accurately represent’
L330: Quantify ‘shows a good agreement’
L330: remove sentence ‘As a result, …’
L332: COT repeated twice
Figure 3: How much was the COT and CER tuned to agree? Could we get the right answer for the wrong reason?
L341: how did you arrive a 11 wavelengths? What happens if you use 10 or 12? Aka, how sensitive are you to this choice? Would be a great opportunity to plot accuracy vs number of wavelengths.
Figure 5: how do the other bands look like? 5 a) looks very noisy far away from the clouds.
L463: ‘bands, potentially increasing’
L483: state footprint sizes of upcoming satellites, name and cite those satellites
L487: why did you not investigate smaller footprints?
L490: ‘of pronounced biases’
L490-491: not clear what changes are not significant
L495: ‘In conclusion, future satellite missions with any …’ That is a very strong statement without any quantification. This would depend on the retrieval algorithm, chosen bands, accuracy requirements, area of interest, …
L500: quantify ‘to substantial 3D’
L501: why do 3D cloud biases need to be considered in the initial planning stage? Algorithms are typically tackled much later.
L517: How could the bypass method deal with cloud shadows?
L524: Quality Flag =0 or 1 are not ‘best quallity’ data. That would only by 0
L524: How are the values in Table 2 derived for the bypass method when they don’t include 3D RT calculations.
Figure 9: Not sure if b) is improved compared to a) outside of the cloud shadow area.
L570 – L573: You state a problem with thin or partial clouds for the bypass method. How would an operational algorithm deal with that?
L585: remove ‘on a cluster at the University of Colorado’
L592: You state that the bypass method can be supplemented by periodic full calculations. How would that work in detail? When do you run them, how do you use their results to improve the results?
Figure 12. Where does the XCO2 in those scenes come from?
L630: ‘We documented the …’ -> The main manuscript does not contain any documentation of the toolbox. Would remove that statement.
L671: ‘more accurate level of accuracy’?
L672: remove last sentence ‘It will improve …’ Your study did not show any information to draw that conclusion.
L685: GitHub for OCO-2 toolbox leads to a 404 page not found
Figure A3. Why is cloud effective radius only one fixed number for the whole scene?
Figure A6. Would remove. There is not much information here beyond what one would expect.
Citation: https://doi.org/10.5194/egusphere-2024-1936-RC2 -
AC3: 'Reply on RC2', Yu-Wen Chen, 14 Nov 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-1936/egusphere-2024-1936-AC3-supplement.pdf
-
AC3: 'Reply on RC2', Yu-Wen Chen, 14 Nov 2024
Viewed
HTML | XML | Total | BibTeX | EndNote | |
---|---|---|---|---|---|
310 | 81 | 172 | 563 | 11 | 12 |
- HTML: 310
- PDF: 81
- XML: 172
- Total: 563
- BibTeX: 11
- EndNote: 12
Viewed (geographical distribution)
Country | # | Views | % |
---|
Total: | 0 |
HTML: | 0 |
PDF: | 0 |
XML: | 0 |
- 1