the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 31 Aug 2026
Physically Structured Target Design Improves the Robustness and Transferability of Neural Network Methane Retrievals
Abstract. Artificial neural networks (ANNs) offer a computationally efficient alternative to conventional physics-based satellite retrievals of atmospheric methane (CH₄). However, the impact of target design on retrieval generalization and robustness remains unclear. Here, we develop two ANN-based column-averaged dry-air mole fraction of methane (XCH₄) retrieval frameworks from GOSAT-2 observations. The models are trained with full-physics (ANN-FP) or proxy (ANN-Proxy) retrieval products and evaluated against 2021–2022 observations and TCCON measurements. While both ANNs reproduce their training targets during training, ANN-Proxy is more stable out of sample, whereas ANN-FP shows increasing bias and drift. Mechanistic analyses using feature attribution, local gradient geometry, and environmentally conditioned perturbation modes indicate that ANN-Proxy concentrates over 97 % of its attribution within CH₄ and carbon dioxide (CO₂) absorption windows and maintains a coherent sensitivity field (mean gradient-direction similarity 0.96 versus 0.72 for ANN-FP). Furthermore, ANN-Proxy sensitivity directions are less aligned with aerosol- and albedo-induced perturbation modes, indicating reduced environmental coupling. The proxy-oriented target shapes the learned mapping to enhance robustness and transferability. These results suggest that physically structured proxy targets provide a practical strategy for rapid, stable, and operationally deployable neural-network methane retrievals in future carbon-monitoring satellite missions.
- Preprint
(1853 KB) - Metadata XML
- BibTeX
- EndNote
Status: final response (author comments only)
- RC1: 'Comment on egusphere-2026-3549', Anonymous Referee #1, 15 Sep 2026
-
RC2: 'Comment on egusphere-2026-3549', Anonymous Referee #2, 27 Oct 2026
This manuscript by Song et al. evaluates two artificial neural network (ANN)-based emulators, each using full-physics (FP) and proxy GOSAT-2 XCH4 retrievals as the target. This work clearly shows that the ANN model trained by the proxy XCH4 retrieval outperforms the one trained by the FP retrieval. Various analysis of the ANN models (SHAP, gradient geometry, responds to perturbations) support the main conclusion from different angles. The authors did well in making the manuscript concise with a clear main message. The results contribute to the ongoing community effort to speed up the computationally expensive greenhouse gas retrievals, especially by leveraging the recent advancement of deep learning/AI. Hopefully, the following comments will further improve this work.
The manuscript will benefit from more detailed description and justification of the traditional retrievals (FP and proxy) and the ANN architectures that are trained on them. In Section 3.1, the authors should cite the original papers/ATBDs for the FP and proxy retrievals. It is important to provide the state vector elements (e.g., trace gas layers, temperature, surface pressure, albedo parameterization, scatterers, etc.) and static inputs such as priors, meteorology, and spectroscopy. Each retrieval is a function that maps the inputs into an output (XCH4 for FP and R for proxy), and the ANN models essentially fit these two functions, so it is crucial to adequately present the original functions. Section 3.1 should also clearly provide the inputs (e.g., what, source, dimension, and any difference from the retrieval) to the ANN. These can be inferred from Fig. 7 but should be clarified in text earlier.
Deep neural networks have proven to be extremely capable of approximating high-dimensional, nonlinear functions. That capability is unlocked by large training datasets and rich feature space. It is unclear why this work only uses a small subset for training – half in 2020 because of the even train/validation split. The much larger portion of 2021-2022 data are all used in out-of-sample testing. There is not much information about the training process, e.g., what the training/validation loss tells about the model capacity vs. data size and the detailed optimization procedure.
Related to the comment about model capacity and data size, the ANN fit to the FP is not as good as proxy (MAE 6-7 ppb vs ~4 ppb). Would that be caused by different over/underfitting by the two ANN models? What are the inherent XCH4 variabilities for FP and proxy, e.g., over small homogenous areas? The FP is a more challenging function for the ANN to fit, so underfitting is possible even the ANN-FP model is apparently larger.
Minor points:
Title: “Physically structured target design” is a bit hard to understand. It is more of a choice, fitting the CH4/CO2 ratio vs. XCH4 directly, than a design. It is also unclear why one is more “physically structured” than the other.
Line 35: define FP in the main text. Line 46: standardize FP vs. physics-based.
Line 54: LeCun et al. 2015 may not be a proper citation here. It supports deep learning can approximate highly complicated functions with large training sets, but not specifically for CH4 and CO2 retrievals.
Figure 2 and section 3.1: Figure 2 implies there is only one linear layer per block, the text says multiple. The activation function needs to be checked for consistency. There are a large number of hyperparameters in the architecture design (multilayer perceptron vs. other deep learning architectures), number of layers/hidden dimension, activation and normalization (if any). Clarify what are optimized through validation performance and what are assumed/adopted from (what) prior work.
Line 144: are the inference results retrained using the entire 2020 data (merging training and validations sets)?
Figure 4: use lower case r for Pearson correlation coefficient as R is the targeted ratio.
Line 228: need more quantitative results to support the claim that ANN-proxy agrees better with TCCON. Here it just says ANN-proxy correlates better in 4 out of 12 sites.
Figure 6: markers and fonts are too small.
Lines 231-232: the difference between ANN-FP and the FP retrieval target is larger than ANN-proxy vs. the proxy target. That point needs to be made clear with the consideration of the capacity of the ANN-FP model architecture.
Figure 7: explain SHAP value vs. feature value.
Lines 260-264: explain what Lipschitz analysis does and what ||Delta y||/||Delta x|| is in Fig. 8a. Cite the original papers that proposed similar analysis for ANN. I cannot get how the broader distribution by ANN-proxy supports the claims (ANN-proxy relies more on radiances, gradients confined in absorption subspace).
Figure 8: horizontally arrangement of panels may be better than vertical. Explain “H” and the grey shading in panel a. Explain how exactly the Delta RMSE is calculated in panel b. Explain how exactly “cosine similarity of gradients” are calculated “across samples”.
Line 284: explain how exactly “aerosol perturbations” are applied. It appears to be a principal component of an ensemble of difference spectra according to Section 3.2, but it is unclear how it is used in the experiment.
Line 304: “manifolds” only appears once here. Explain what it is and how it is connected to the previous analyses.
Line 320: explain “Jacobian” here. Is it different from “gradient”?
Line 338: please cite relevant MethaneSAT publications.
Citation: https://doi.org/10.5194/egusphere-2026-3549-RC2
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 315 | 151 | 61 | 527 | 69 | 59 |
- HTML: 315
- PDF: 151
- XML: 61
- Total: 527
- BibTeX: 69
- EndNote: 59
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Review of “Physically Structured Target Design Improves the Robustness and Transferability of Neural Network Methane Retrievals” submitted for publication to AMT by Rongjin Son et al.
This manuscript investigate the impact of the training target on the robustness of neural-network XCH₄ retrievals from GOSAT-2 observations. The authors have compared an ANN trained to reproduce the full-physics (FP) retrieval with a proxy-oriented framework based on the CH₄/CO₂ ratio. They show substantially better stability for the proxy-oriented approach, and complement the retrieval evaluation with TCCON comparisons and several analyses of the learned models, including SHAP attribution, gradient geometry, and empirical aerosol- and albedo-related perturbation modes.
The manuscript is very clearly written, and I have not identify any flaw that would invalidate the empirical results. The degradation of ANN-FP relative to ANN-Proxy when moving from the 2020 training period to 2021–2022 appears quite clear. However, I have some reservations regarding the interpretation of the comparisons. I do not think that the experimental design fully supports the central conclusion that the improved robustness results from the choice of the training target itself.
My main concern is that the two experiments differ in several respects simultaneously. ANN-FP and ANN-Proxy do not simply use different supervised targets: they also have different architectures, different information pathways, and different sets/organization of input variables. In addition, ANN-Proxy predicts the CH₄/CO₂ ratio, from which XCH₄ is subsequently reconstructed using a model-based XCO₂ field. The comparison therefore contrasts two retrieval frameworks rather than two otherwise identical neural networks differing only in their target. The authors themselves acknowledge this issue in Section 4.2, noting that the frameworks differ “not only in their supervised targets but also in the information pathways emphasized during learning,” and later state that the observed behavior may arise from “the proxy-oriented retrieval framework as a whole rather than from the retrieval target alone.”
This distinction is important because the title, abstract, and conclusions place considerable emphasis on target design as the mechanism responsible for the improved robustness. A more convincing demonstration would require an ablation experiment in which the FP and proxy targets are learned using the same architecture and the same input information. Ideally, target choice and architecture/input design would be varied independently. Without such an experiment, it seems difficult to determine how much of the observed improvement originates from the proxy target itself and how much results from the architecture and information content provided to the two networks.
This issue also affects the interpretation of the mechanistic analyses. For example, the fact that ANN-Proxy concentrates more than 97% of its SHAP attribution in the CH₄ and CO₂ absorption bands is presented as evidence that the proxy target leads to a more physically constrained representation. However, ANN-Proxy is explicitly constructed around separate CH₄ and CO₂ spectral branches and has a different set of auxiliary inputs from ANN-FP. Therefore, at least part of the difference in feature attribution may be imposed by the model design rather than emerging as a consequence of the target. The same caution applies, I think, to the interpretation of the gradient-space analyses.
More generally, I am not entirely convinced that the principal empirical result is unexpected. The motivation for proxy retrievals is precisely that the CH₄/CO₂ ratio reduces sensitivity to common light-path perturbations. The manuscript itself explains that this cancellation is one of the main advantages of the proxy approach. It is therefore not surprising that a neural network designed and trained to emulate such a retrieval exhibits less sensitivity to aerosol and surface-reflectance variability than a network emulating a full-physics product. I think the manuscript would benefit from a clearer discussion of what is genuinely learned from the present experiments beyond this expected property of proxy retrievals.
Finally, the claims regarding operational or onboard applicability appear somewhat stronger than what is demonstrated here. ANN-Proxy is indeed smaller and faster than ANN-FP, but the quoted inference time concerns the neural network itself. The final XCH₄ retrieval additionally requires a model-based XCO₂ field. This external information requirement should be considered when discussing onboard implementation, particularly since the manuscript itself notes that proxy retrievals depend on the accuracy of the CO₂ reference field. Also, there is no indication that onboard processing power is really the main issue in the context of methane observation.
Overall, I find the empirical comparison potentially useful, but I think the conclusions need to be more carefully aligned with what the experimental design actually demonstrates. At present, the results provide convincing evidence that the proxy-oriented retrieval framework as implemented here is more stable under temporal extrapolation than the implemented ANN-FP framework. They do not, in my view, yet demonstrate that target design itself is responsible for this improvement. An appropriate set of ablation experiments, or alternatively a substantial reframing of the conclusions around the comparison of the two complete retrieval frameworks, would correct the main weakness of the manuscript.
Technical comment
Line 34 “particularly in FP retrieval”. Is there any indication that there are less systematic errors for other types of retrievals ?
Line 54 : I do not think that this reference is realy appropriate for the statement
Figure 1 : Color scale should be white for zero and colorized for 1 or larger. Do not have white for ≈50 samples.
Also, it would make sense to have different ranges for the two color scales to judge where the spatial distributions are exactly the same, or not. If the distributions are exactly the same (which is expected for random sampling), a single figure is sufficient
Figure 6 : Please provide more explanation on the figure. What are the vertical bars ? How are the days selected ?