the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 08 Sep 2022
Detecting and quantifying methane emissions from oil and gas production: algorithm development with ground-truth calibration based on Sentinel-2 satellite imagery
Zhan Zhang
Evan D. Sherwin
Daniel J. Varon
Adam R. Brandt
Abstract. Sentinel-2 satellite imagery has been shown by studies to be capable of detecting and quantifying methane emis- sions from oil and gas production. However, current methods lack performance validation by calibration with ground-truth testing. This study developed a multi-band-multi-pass-multi-comparison methane retrieval algorithm that enhances Sentinel-2 sensitivity to methane plumes. The method was calibrated using data from a large-scale controlled release test in Ehrenberg, Arizona in fall 2021, with three algorithm parameters tuned based on the true emission rates. Tuned parameters are the pixel- level concentration upper bound threshold during extreme value removal, the number of comparison dates, and the pixel-level methane concentration percentage threshold when determining the spatial extent of a plume. We found that a low value of the upper bound threshold during extreme value removal can result in false negatives. A high number of comparison dates helps enhance the algorithm sensitivity to the plumes in the target date, but values in excess of 12 days are neither necessary nor computationally efficient. A high percentage threshold when determining the spatial extent of a plume helps enhance the quan- tification accuracy, but it may harm the yes/no detection accuracy. We found that there is a trade-off between quantification accuracy and detection accuracy. In a scenario with the highest quantification accuracy, we achieved the lowest quantification error and had zero false positive detections; however, the algorithm missed 3 true plumes which reduced the yes/no detection accuracy. On the contrary, all the true plumes were detected in the highest detection accuracy scenario, but the emission rate quantification had higher errors. We also illustrated a two-step method that updates the emission rate estimates in an interim step which improves quantification accuracy while keeping high yes/no detection accuracy. We also validated the algorithm’s ability to avoid false positives by applying it to a nearby region with no emissions.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(35503 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(35503 KB) - BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Zhan Zhang et al.
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2022-771', Anonymous Referee #1, 19 Sep 2022
The authors developed a multi-band-multi-pass-multi-comparison methane retrieval algorithm that enhances Sentinel-2 sensitivity to methane plumes. The new algorithm is based on the algorithm developed by the same author but enhances its sensitivity to methane plumes and reduces false detections. The manuscript is well written. The method looks sound. I recommend publication after minor revision.
General comments:
- Section 3.1. Line 250-259. What is the relationship between “two-step application” and MBMPMC? It is not clear to me.
- Figure 7. The improvement compared to the previous 3 methods are very impressive. Please try to briefly summarize the reasons for the improvement compared to each method.
Specific comments:
- There are two “also” in the last two sentences. Please try to rephrase them.
- It is useful to clarify the ratio of anthropogenic to natural emissions as well.
- Section 2.1. Please mention that there is a flow chart to illustrate the steps of MBMPMC when first discussing them.
Citation: https://doi.org/10.5194/egusphere-2022-771-RC1 - AC2: 'Reply on RC1', Zhan Zhang, 28 Sep 2022
- AC1: 'Comment on egusphere-2022-771', Zhan Zhang, 28 Sep 2022
-
RC2: 'Comment on egusphere-2022-771', Anonymous Referee #2, 30 Sep 2022
The manuscript by Zhang et al. deals with methane plume retrievals with the Sentinel-2 satellite mission. They use methane concentration enhancement maps derived from Sentinel-2 data over controlled methane releases to constrain free parameters in the retrieval and emission rate estimation algorithms. They show that the Sentinel-2 detection and quantification of methane plumes from those controlled releases improves after model calibration with the same in situ data.
In my opinion, the research discussed in this manuscript must be of interest to the methane remote sensing community, especially considering the recent and rapid development of satellite-based high-resolution methane mapping methods. Also, the topic fits perfectly in AMTD, where the first paper on the use of Sentinel-2 for methane mapping (Varon et al., 2021) was published.
On the other hand, I have some major concerns with the overall purpose and some technical details of this work. In particular, I am not sure about the value of calibrating the algorithms with ground truth in this case. Is there any hope that they can be extrapolated to other sites or even seasons at the same site? I would say no. The algorithm parameters that they are optimizing are strongly acquisition dependent. For example, the thresholds accounting for outliers and false positives are driven by surface characteristics (homogeneity, stability). The finding that 12 dates are optimal for the multitemporal method wouldn’t apply to a site with changing vegetation covers, for which a configuration with one recent reference acquisition would be better than with a combination of 12 of them. Also, the thresholds used to filter out outliers should depend on the heterogeneity of the site. The retrieval noise, and hence the spatial extent of the plume, will also depend on the surface heterogeneity.
This extrapolation question would also apply to the proposed two-step method to improve emission quantification. Is this relevant to a wide community if data from a controlled methane release are needed as input?
I think that the authors should show that the estimated model parameters can be applied outside this particular experiment for this work to be relevant. I don’t think that the no false-negative test in Fig. 8 is a proper assessment of the model extrapolation that I am asking for (no emission to evaluate, and acquisition conditions for site B are very close to those of site A).
Perhaps the authors could run tests of how those thresholds perform for other sites, especially those with a more complex surface such as the US sites included in Ehret et al. https://pubs.acs.org/doi/10.1021/acs.est.1c08575?
In addition, I think it should it be possible to use this nice dataset to investigate possible approaches for automatic estimation of the outlier filtering threshold and the plume definition threshold. Those thresholds should be based on scene-based noise/heterogeneity estimates, such as n-sigmas above the retrieval noise level. Perhaps the authors could come up with approaches to estimate those parameters from the methane enhancement maps and cross-compare the results with the values derived from the model calibration presented in the manuscript. Examples of such threshold estimation approaches can be found in Ehret et al. (Background Estimation).
Other comments:
Sec 3.1, List of steps to improve plume detection: most of those steps (clear-view overpasses, normalization, removal of outliers, multiple reference data) are relatively obvious and already included in existing algorithms (e.g. Ehret et al., Gorrono et al.). Because of this, I am not sure that the methodology presented in this manuscript deserves to be presented as a new method, including an own acronym.
L2. (and L159) “performance validation by calibration” – not sure what this means
L288 – Not sure about statements on thresholds and detection accuracy: comments might apply to the avoidance of false negatives, but not the occurrence of false positives (which is actually the main difficulty for plume detection in real detection scenarios).
Units: a space is missing between numbers and units, such as in “30m”
L145. Please, check citations.
Shouldn’t this preprint on the Stanford methane release experiment be cited https://eartharxiv.org/repository/view/3465/?
Citation: https://doi.org/10.5194/egusphere-2022-771-RC2 - AC3: 'Reply on RC2', Zhan Zhang, 11 Nov 2022
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2022-771', Anonymous Referee #1, 19 Sep 2022
The authors developed a multi-band-multi-pass-multi-comparison methane retrieval algorithm that enhances Sentinel-2 sensitivity to methane plumes. The new algorithm is based on the algorithm developed by the same author but enhances its sensitivity to methane plumes and reduces false detections. The manuscript is well written. The method looks sound. I recommend publication after minor revision.
General comments:
- Section 3.1. Line 250-259. What is the relationship between “two-step application” and MBMPMC? It is not clear to me.
- Figure 7. The improvement compared to the previous 3 methods are very impressive. Please try to briefly summarize the reasons for the improvement compared to each method.
Specific comments:
- There are two “also” in the last two sentences. Please try to rephrase them.
- It is useful to clarify the ratio of anthropogenic to natural emissions as well.
- Section 2.1. Please mention that there is a flow chart to illustrate the steps of MBMPMC when first discussing them.
Citation: https://doi.org/10.5194/egusphere-2022-771-RC1 - AC2: 'Reply on RC1', Zhan Zhang, 28 Sep 2022
- AC1: 'Comment on egusphere-2022-771', Zhan Zhang, 28 Sep 2022
-
RC2: 'Comment on egusphere-2022-771', Anonymous Referee #2, 30 Sep 2022
The manuscript by Zhang et al. deals with methane plume retrievals with the Sentinel-2 satellite mission. They use methane concentration enhancement maps derived from Sentinel-2 data over controlled methane releases to constrain free parameters in the retrieval and emission rate estimation algorithms. They show that the Sentinel-2 detection and quantification of methane plumes from those controlled releases improves after model calibration with the same in situ data.
In my opinion, the research discussed in this manuscript must be of interest to the methane remote sensing community, especially considering the recent and rapid development of satellite-based high-resolution methane mapping methods. Also, the topic fits perfectly in AMTD, where the first paper on the use of Sentinel-2 for methane mapping (Varon et al., 2021) was published.
On the other hand, I have some major concerns with the overall purpose and some technical details of this work. In particular, I am not sure about the value of calibrating the algorithms with ground truth in this case. Is there any hope that they can be extrapolated to other sites or even seasons at the same site? I would say no. The algorithm parameters that they are optimizing are strongly acquisition dependent. For example, the thresholds accounting for outliers and false positives are driven by surface characteristics (homogeneity, stability). The finding that 12 dates are optimal for the multitemporal method wouldn’t apply to a site with changing vegetation covers, for which a configuration with one recent reference acquisition would be better than with a combination of 12 of them. Also, the thresholds used to filter out outliers should depend on the heterogeneity of the site. The retrieval noise, and hence the spatial extent of the plume, will also depend on the surface heterogeneity.
This extrapolation question would also apply to the proposed two-step method to improve emission quantification. Is this relevant to a wide community if data from a controlled methane release are needed as input?
I think that the authors should show that the estimated model parameters can be applied outside this particular experiment for this work to be relevant. I don’t think that the no false-negative test in Fig. 8 is a proper assessment of the model extrapolation that I am asking for (no emission to evaluate, and acquisition conditions for site B are very close to those of site A).
Perhaps the authors could run tests of how those thresholds perform for other sites, especially those with a more complex surface such as the US sites included in Ehret et al. https://pubs.acs.org/doi/10.1021/acs.est.1c08575?
In addition, I think it should it be possible to use this nice dataset to investigate possible approaches for automatic estimation of the outlier filtering threshold and the plume definition threshold. Those thresholds should be based on scene-based noise/heterogeneity estimates, such as n-sigmas above the retrieval noise level. Perhaps the authors could come up with approaches to estimate those parameters from the methane enhancement maps and cross-compare the results with the values derived from the model calibration presented in the manuscript. Examples of such threshold estimation approaches can be found in Ehret et al. (Background Estimation).
Other comments:
Sec 3.1, List of steps to improve plume detection: most of those steps (clear-view overpasses, normalization, removal of outliers, multiple reference data) are relatively obvious and already included in existing algorithms (e.g. Ehret et al., Gorrono et al.). Because of this, I am not sure that the methodology presented in this manuscript deserves to be presented as a new method, including an own acronym.
L2. (and L159) “performance validation by calibration” – not sure what this means
L288 – Not sure about statements on thresholds and detection accuracy: comments might apply to the avoidance of false negatives, but not the occurrence of false positives (which is actually the main difficulty for plume detection in real detection scenarios).
Units: a space is missing between numbers and units, such as in “30m”
L145. Please, check citations.
Shouldn’t this preprint on the Stanford methane release experiment be cited https://eartharxiv.org/repository/view/3465/?
Citation: https://doi.org/10.5194/egusphere-2022-771-RC2 - AC3: 'Reply on RC2', Zhan Zhang, 11 Nov 2022
Peer review completion
Journal article(s) based on this preprint
Zhan Zhang et al.
Zhan Zhang et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 293 | 202 | 16 | 511 | 2 | 4 |
- HTML: 293
- PDF: 202
- XML: 16
- Total: 511
- BibTeX: 2
- EndNote: 4
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(35503 KB) - Metadata XML