the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 02 May 2022
Bayesian atmospheric correction over land: Sentinel-2/MSI and Landsat 8/OLI
Abstract. Mitigating the impact of atmospheric effects on optical remote sensing data is critical for monitoring intrinsic land processes and developing Analysis Ready Data (ARD). This work develops an approach to this for the NERC NCEO medium resolution ARD Landsat 8 (L8) and Sentinel 2 (S2) products, called Sensor Invariant Atmospheric Correction (SIAC). The contribution of the work is to phrase and solve that problem within a probabilistic (Bayesian) framework for medium resolution multispectral sensors S2/MSI and L8/OLI and provide per-pixel uncertainty estimates traceable from assumed top-of-atmosphere (TOA) measurement uncertainty, making progress towards an important aspect of CEOS ARD target requirements.
A set of observational and a priori constraints are developed in SIAC to constrain an estimate of coarse resolution (500 m) Aerosol Optical Thickness (AOT) and Total Column Water Vapour (TCWV), along with associated uncertainty. This is then used to estimate the medium resolution (10–60 m) surface reflectance and uncertainty, given an assumed uncertainty of 5 % in TOA reflectance. The coarse resolution a priori constraints used are: the MODIS MCD43 BRDF/Albedo product giving a constraint on 500 m surface reflectance; and Copernicus Atmosphere Monitoring Service (CAMS) operational forecasts of AOT and TCWV providing estimates of atmospheric state at core 40 km spatial resolution with an associated 500 m resolution spatial correlation model. The mapping in spatial scale between medium resolution observations and the coarser resolution constraints is achieved using a calibrated effective Point Spread Function for MCD43. Efficient statistical approximations (emulators) to outputs of the 6S atmospheric radiative transfer code used to estimate the state parameters and in the atmospheric correction.
SIAC is demonstrated for a set of global S2 and L8 images covering AERONET and RadCalNet sites. AOT retrievals show a very high correlation to AERONET estimates (R around 0.86, RMSE of 0.07 for both sensors), although with a small bias in AOT. TCWV is accurately retrieved from both sensors (R > 0.96, RMSE < 0.32 g /cm2). Comparisons with in situ surface reflectance measurements from the RadCalNet network show that SIAC provides accurate estimates of surface reflectance across the entire spectrum, with RMSE mismatches with the reference data between 0.01 and 0.02 in units of reflectance, for both S2 and L8. For near-simultaneous S2 and L8 acquisitions, there is a very tight relationship (R > 0.95 for all common bands) between surface reflectance from both sensors, with negligible biases. Uncertainty estimates are assessed through discrepancy analysis and found to provide viable estimates for AOT and TCWV. For surface reflectance, they give conservative estimates of uncertainty, suggesting that a lower estimate of TOA reflectance uncertainty might be appropriate.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(53061 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(53061 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2022-170', Anonymous Referee #1, 09 Jun 2022
General comments:
Atmospheric correction is an essential part of satellite remote sensing of land surface. Yin et al. describe and evaluate the Sensor Invariant Atmospheric Correction (SIAC) algorithm for atmospheric correction. The existing atmospheric correction methods can be improved and therefore further development of the algorithms is welcome.
SIAC is capable of carrying out atmospheric correction both for Sentinel 2 and Landsat 8 satellite data. Furthermore, SIAC is a Bayesian (statistical) algorithm so it can take advantage of prior information and it produces uncertainty estimates for the surface reflectances produced. The algorithm is tested and validated with ground-based AERONET and RadCalNet data. There are no significant steps taken in new method development in this work, but SIAC combines well existing methods. The results shown in the manuscript show that SIAC is capable of carrying out good quality atmospheric correction.
The selections made in the development of SIAC are mostly well justified and based on previously published literature. It is very good that the authors have shared the codes for others to be used.
My main criticism is in the presentation. The quality of presentation in the manuscript varies. Time to time the text is well written and smooth but quite often the text is difficult to read. The manuscript is quite long and structured so that first SIAC and results are explained in general terms followed by the Discussion and Conclusions, and then all the technical details are mostly included in the Appendices. As this is a method development manuscript, I find this a bit difficult for the reader as it is needed to browse back and forth while reading. Furthermore, the manuscript heavily relies on use of acronyms and symbols. It is good that the main symbols are explained in a table but this also makes the manuscript very slow to read, especially for a person who is not that familiar with the field. I would strongly recommend the authors to think the use of acronyms (even single letter acronyms) and symbols, and possibly shorten and re-organise the manuscript for improved readability. Also, there were some typos in the text so proofreading is recommended. Most of the font sizes in figures are too small and very difficult to read.
Specific comments:
l.15 Abstract mentions efficient emulators. However, in the text this was a bit unclear how these were used. Could it be clarified?
l.53 AERONET is based on remote sensing, not in-situ measurements.
l.83 ".., we calculate:". "Calculate atmospheric parameter estimation"?
l.84 The list is difficult to follow.
l.129 In the abstract this was probably mentioned as "statistical emulation", neural networks are not really statistical emulation.
l.133 "This may cause errors". Can you estimate the significance of these errors
l.169 "Matrix D" was this defined?
l.183 "We assume that mean atmospheric parameters...are constant..." Can you estimate the significance of this assumption?
l.221 Can you give an example of "other artifacts"
l.223 Why the tolerance of 10% was selected? How this filtering affects the evaluation of the results?
l.300 "corrected to R", what is R?
l.333 Why different TCWV gamma values are used for S2 and L8?
p.23 Figure 13. What are the colors of the bars?
l.374 Is IQR defined?
l.375 Validation of surface reflectance uncertainty was a bit unclear. Should be clarified.
l.390 In many sentences (especially on page 25) you use "that" & "this" and it is unclear to which word these are referring to
p.26 Figure 15. What are the colors?
Citation: https://doi.org/10.5194/egusphere-2022-170-RC1 - AC2: 'Reply on RC1', Feng Yin, 05 Jul 2022
-
CEC1: 'Comment on egusphere-2022-170', Juan Antonio Añel, 15 Jun 2022
Dear authors,
After checking your manuscript, it has come to our attention that it does not comply with our Code and Data Policy.
https://www.geoscientific-model-development.net/policies/code_and_data_policy.html
You have archived your code on GitHub. However, GitHub is not a suitable repository. GitHub itself instructs authors to use other alternatives for long-term archival and publishing, such as Zenodo. Therefore, please, publish your code in one of the appropriate repositories, and include the relevant primary input/output data. In this way, you must include in a potential reviewed version of your manuscript the modified 'Code and Data Availability' section, the DOI of the code (and another DOI for the dataset if necessary).Please, reply as soon as possible to this comment with the new links and DOIs, and be aware that failing to comply with this request will result in the rejection of your manuscript.
Juan A. Añel
Geosci. Model Dev. Exec. EditorCitation: https://doi.org/10.5194/egusphere-2022-170-CEC1 -
AC1: 'Reply on CEC1', Feng Yin, 16 Jun 2022
Dear Juan,
Thanks for pointing out the issue. I have uploaded the code into Zenodo along with the validaiton dataset used in this manuscript. The DOI to the code: https://doi.org/10.5281/zenodo.6651964 and the validation datasets: https://doi.org/10.5281/zenodo.6652892.
Best regards,
Feng.
Citation: https://doi.org/10.5194/egusphere-2022-170-AC1
-
AC1: 'Reply on CEC1', Feng Yin, 16 Jun 2022
-
RC2: 'Comment on egusphere-2022-170', Hankui Zhang, 24 Jun 2022
I reviewed the manuscript years ago submitted to another journal + I did look forwarding to the revision which I did not see. Now I saw it in a different journal with significantly improvement in the experimental design and presentation style.
I recommended for publication as the study
- presented a novel way for medium resolution (Landsat-8 and Sentinel-2) satellite data atmospheric correction
- showed the presented method significantly improves the surface reflectance and aerosol property retrieval accuracy
- systematically validated the presented method using Landsat and Sentinel-2 data over globally distributed AERONET
Impressive work and I have to say this is a hard paper to review as there are many techniques used in the study. I appreciated the authors make the codes public available. I like many ideas in the paper but in particular the spatial smoothness prior and the validation of the uncertainty (many geostationary satellite Bayesian AOD estimation output uncertainty without validation). I really hope the authors later (not in this manuscript) can experimentally show that the proposed method is better than LaSRC and Sen2Cor – by running LaSRC and Sen2Cor software on the datasets used in this study (this maybe boring to the authors but I believe it is beneficial to the community and to the authors if the algorithm more convinced). Or simply participate ACIX and hope to see the algorithm will be an unambiguous winner in ACIX.
I have a few comments before the publication of the manuscript
Major issues
1, The readers may wonder what is the computational cost of the proposed method and the iteration nature of the gradient descent (L-BFGS-B algorithm) may boost the computation time rapidly. Add a paragraph for discussion.
2, The authors omitted the deep blue bands for both Landsat-8 and Sentinel-2 and the band is most sensitive to aerosol among all the bands, any particular reason for doing so. MODIS does not have this band? use MODIS blue to replace not work ?
3, Spatial smoothness is interesting but unclear mostly related to how D matrix is derived. Is the spatial difference applied to the intermediate AOD and WV results derived for each iteration of L-BFGS-B algorithm. If so, Cxb is changing with iterations? The spatial difference is calculated over neighbor 8 pixels or over an entire 40 km window? How the row and column direction are averaged? Simply give an example of deriving D would solve my concern. After the line of 775, “over the whole of Gc” or of sub-40 km? Looks like the sub-40 km is used as a block to solve the equation which means the solution of each sub-40 km is independent to the neighbor sub-40 km block (but I am not sure).
The study need a consistent check of the writing style. For example, lines 102-103.
Minor issues
Discuss the possible shift from MODIS to VIIRS considering the dying of MODIS.
Line 105, how to treat b09. Is MODIS band 2 reflectance (after D2 transformation) used as a prior for b09 reflectance?
Line 675, there is another MCD43 PSF estimation paper worth citing (Che et al. 2021).
Che, X., Zhang, H. K., & Liu, J. (2021). Making Landsat 5, 7 and 8 reflectance consistent using MODIS nadir-BRDF adjusted reflectance as reference. Remote Sensing of Environment, 262, 112517.
Line 130, not quite familiar with the L-BFGS-B algorithm. But does the L-BFGS-B algorithm have its own way to calculate the gradient or the authors really used the back propagation of ANN to calculate ANN (ANN uses an automatic differentiation method) to feed into L-BFGS-B algorithm. Or back prorogation gradient is only used in B4 for RBF uncertainty but not used in L-BFGS-B?
Line 140, the LaSRC cannot be considered as DDV method anymore as each single pixel on the global have a visible/SWIR ratio (not just DDV pixels) see Vermote et al. 2016.
Line 155, the framework can tolerate incomplete coverage of Y^, how? If incomplete Y^, then the incomplete inversion of aod and wv will be spatially interpolated? Or get value using spatial smooth prior.
Line 200, “a estimates” – grammar
Line 260, how the σxaero is set
Line 310, which TOA band, the blue band should definitely greater than 10% mostly
Line 330 we are already in Section 4.3
Please label and re-arrange figures 10-12 correctly. Some are clearly surface but labelled as TOA. Check the labels of the plots. Rearrange them so that the TOA spectra and TOA plot are at the same row.
Line 470, yes, in many Landsat calibration literature, the calibration accuracy is 3%. Should this linked to appendix H?
Line 580. There is no Schaff 2021 in reference list
Citation: https://doi.org/10.5194/egusphere-2022-170-RC2 - AC3: 'Reply on RC2', Feng Yin, 05 Jul 2022
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2022-170', Anonymous Referee #1, 09 Jun 2022
General comments:
Atmospheric correction is an essential part of satellite remote sensing of land surface. Yin et al. describe and evaluate the Sensor Invariant Atmospheric Correction (SIAC) algorithm for atmospheric correction. The existing atmospheric correction methods can be improved and therefore further development of the algorithms is welcome.
SIAC is capable of carrying out atmospheric correction both for Sentinel 2 and Landsat 8 satellite data. Furthermore, SIAC is a Bayesian (statistical) algorithm so it can take advantage of prior information and it produces uncertainty estimates for the surface reflectances produced. The algorithm is tested and validated with ground-based AERONET and RadCalNet data. There are no significant steps taken in new method development in this work, but SIAC combines well existing methods. The results shown in the manuscript show that SIAC is capable of carrying out good quality atmospheric correction.
The selections made in the development of SIAC are mostly well justified and based on previously published literature. It is very good that the authors have shared the codes for others to be used.
My main criticism is in the presentation. The quality of presentation in the manuscript varies. Time to time the text is well written and smooth but quite often the text is difficult to read. The manuscript is quite long and structured so that first SIAC and results are explained in general terms followed by the Discussion and Conclusions, and then all the technical details are mostly included in the Appendices. As this is a method development manuscript, I find this a bit difficult for the reader as it is needed to browse back and forth while reading. Furthermore, the manuscript heavily relies on use of acronyms and symbols. It is good that the main symbols are explained in a table but this also makes the manuscript very slow to read, especially for a person who is not that familiar with the field. I would strongly recommend the authors to think the use of acronyms (even single letter acronyms) and symbols, and possibly shorten and re-organise the manuscript for improved readability. Also, there were some typos in the text so proofreading is recommended. Most of the font sizes in figures are too small and very difficult to read.
Specific comments:
l.15 Abstract mentions efficient emulators. However, in the text this was a bit unclear how these were used. Could it be clarified?
l.53 AERONET is based on remote sensing, not in-situ measurements.
l.83 ".., we calculate:". "Calculate atmospheric parameter estimation"?
l.84 The list is difficult to follow.
l.129 In the abstract this was probably mentioned as "statistical emulation", neural networks are not really statistical emulation.
l.133 "This may cause errors". Can you estimate the significance of these errors
l.169 "Matrix D" was this defined?
l.183 "We assume that mean atmospheric parameters...are constant..." Can you estimate the significance of this assumption?
l.221 Can you give an example of "other artifacts"
l.223 Why the tolerance of 10% was selected? How this filtering affects the evaluation of the results?
l.300 "corrected to R", what is R?
l.333 Why different TCWV gamma values are used for S2 and L8?
p.23 Figure 13. What are the colors of the bars?
l.374 Is IQR defined?
l.375 Validation of surface reflectance uncertainty was a bit unclear. Should be clarified.
l.390 In many sentences (especially on page 25) you use "that" & "this" and it is unclear to which word these are referring to
p.26 Figure 15. What are the colors?
Citation: https://doi.org/10.5194/egusphere-2022-170-RC1 - AC2: 'Reply on RC1', Feng Yin, 05 Jul 2022
-
CEC1: 'Comment on egusphere-2022-170', Juan Antonio Añel, 15 Jun 2022
Dear authors,
After checking your manuscript, it has come to our attention that it does not comply with our Code and Data Policy.
https://www.geoscientific-model-development.net/policies/code_and_data_policy.html
You have archived your code on GitHub. However, GitHub is not a suitable repository. GitHub itself instructs authors to use other alternatives for long-term archival and publishing, such as Zenodo. Therefore, please, publish your code in one of the appropriate repositories, and include the relevant primary input/output data. In this way, you must include in a potential reviewed version of your manuscript the modified 'Code and Data Availability' section, the DOI of the code (and another DOI for the dataset if necessary).Please, reply as soon as possible to this comment with the new links and DOIs, and be aware that failing to comply with this request will result in the rejection of your manuscript.
Juan A. Añel
Geosci. Model Dev. Exec. EditorCitation: https://doi.org/10.5194/egusphere-2022-170-CEC1 -
AC1: 'Reply on CEC1', Feng Yin, 16 Jun 2022
Dear Juan,
Thanks for pointing out the issue. I have uploaded the code into Zenodo along with the validaiton dataset used in this manuscript. The DOI to the code: https://doi.org/10.5281/zenodo.6651964 and the validation datasets: https://doi.org/10.5281/zenodo.6652892.
Best regards,
Feng.
Citation: https://doi.org/10.5194/egusphere-2022-170-AC1
-
AC1: 'Reply on CEC1', Feng Yin, 16 Jun 2022
-
RC2: 'Comment on egusphere-2022-170', Hankui Zhang, 24 Jun 2022
I reviewed the manuscript years ago submitted to another journal + I did look forwarding to the revision which I did not see. Now I saw it in a different journal with significantly improvement in the experimental design and presentation style.
I recommended for publication as the study
- presented a novel way for medium resolution (Landsat-8 and Sentinel-2) satellite data atmospheric correction
- showed the presented method significantly improves the surface reflectance and aerosol property retrieval accuracy
- systematically validated the presented method using Landsat and Sentinel-2 data over globally distributed AERONET
Impressive work and I have to say this is a hard paper to review as there are many techniques used in the study. I appreciated the authors make the codes public available. I like many ideas in the paper but in particular the spatial smoothness prior and the validation of the uncertainty (many geostationary satellite Bayesian AOD estimation output uncertainty without validation). I really hope the authors later (not in this manuscript) can experimentally show that the proposed method is better than LaSRC and Sen2Cor – by running LaSRC and Sen2Cor software on the datasets used in this study (this maybe boring to the authors but I believe it is beneficial to the community and to the authors if the algorithm more convinced). Or simply participate ACIX and hope to see the algorithm will be an unambiguous winner in ACIX.
I have a few comments before the publication of the manuscript
Major issues
1, The readers may wonder what is the computational cost of the proposed method and the iteration nature of the gradient descent (L-BFGS-B algorithm) may boost the computation time rapidly. Add a paragraph for discussion.
2, The authors omitted the deep blue bands for both Landsat-8 and Sentinel-2 and the band is most sensitive to aerosol among all the bands, any particular reason for doing so. MODIS does not have this band? use MODIS blue to replace not work ?
3, Spatial smoothness is interesting but unclear mostly related to how D matrix is derived. Is the spatial difference applied to the intermediate AOD and WV results derived for each iteration of L-BFGS-B algorithm. If so, Cxb is changing with iterations? The spatial difference is calculated over neighbor 8 pixels or over an entire 40 km window? How the row and column direction are averaged? Simply give an example of deriving D would solve my concern. After the line of 775, “over the whole of Gc” or of sub-40 km? Looks like the sub-40 km is used as a block to solve the equation which means the solution of each sub-40 km is independent to the neighbor sub-40 km block (but I am not sure).
The study need a consistent check of the writing style. For example, lines 102-103.
Minor issues
Discuss the possible shift from MODIS to VIIRS considering the dying of MODIS.
Line 105, how to treat b09. Is MODIS band 2 reflectance (after D2 transformation) used as a prior for b09 reflectance?
Line 675, there is another MCD43 PSF estimation paper worth citing (Che et al. 2021).
Che, X., Zhang, H. K., & Liu, J. (2021). Making Landsat 5, 7 and 8 reflectance consistent using MODIS nadir-BRDF adjusted reflectance as reference. Remote Sensing of Environment, 262, 112517.
Line 130, not quite familiar with the L-BFGS-B algorithm. But does the L-BFGS-B algorithm have its own way to calculate the gradient or the authors really used the back propagation of ANN to calculate ANN (ANN uses an automatic differentiation method) to feed into L-BFGS-B algorithm. Or back prorogation gradient is only used in B4 for RBF uncertainty but not used in L-BFGS-B?
Line 140, the LaSRC cannot be considered as DDV method anymore as each single pixel on the global have a visible/SWIR ratio (not just DDV pixels) see Vermote et al. 2016.
Line 155, the framework can tolerate incomplete coverage of Y^, how? If incomplete Y^, then the incomplete inversion of aod and wv will be spatially interpolated? Or get value using spatial smooth prior.
Line 200, “a estimates” – grammar
Line 260, how the σxaero is set
Line 310, which TOA band, the blue band should definitely greater than 10% mostly
Line 330 we are already in Section 4.3
Please label and re-arrange figures 10-12 correctly. Some are clearly surface but labelled as TOA. Check the labels of the plots. Rearrange them so that the TOA spectra and TOA plot are at the same row.
Line 470, yes, in many Landsat calibration literature, the calibration accuracy is 3%. Should this linked to appendix H?
Line 580. There is no Schaff 2021 in reference list
Citation: https://doi.org/10.5194/egusphere-2022-170-RC2 - AC3: 'Reply on RC2', Feng Yin, 05 Jul 2022
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 499 | 122 | 9 | 630 | 9 | 7 |
- HTML: 499
- PDF: 122
- XML: 9
- Total: 630
- BibTeX: 9
- EndNote: 7
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Feng Yin
Philip E. Lewis
Jose L. Gómez-Dans
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(53061 KB) - Metadata XML