the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 20 Feb 2026
Quantifying the impact of input data-induced dataset shift on machine learning model applications: A case study of regional reactive nitrogen wet deposition
Abstract. Machine learning (ML) has been extensively applied to studies on spatial distribution characteristics of atmospheric composition, but quantitative assessments of uncertainties arising from input data properties are still lacking. To address this gap, we conducted a case study on wet deposition of reactive nitrogen (Dwet). The Extreme Gradient Boosting (XGBoost) model was employed to predict Dwet in East Asia and Southeast Asia (SEA) with a compiled dataset from multiple sources. We quantified the impacts of input data characteristics on model performance and Dwelt estimations via three sensitivity experiments. Key findings revealed that: (1) Sample size below 6,000–9,000 led to a maximum 12 % accuracy loss, while exceeding this threshold provided marginal performance improvement. (2) Uneven spatial distribution of monitoring sites caused 9–51 % deviations from baseline performance, with >50 % variability in Dwet estimations in data-scarce regions (e.g., western China and SEA). (3) Imbalanced site types lead to insufficient representation of remote sites, resulted in 9–40 % overall accuracy loss and a high risk of severe overestimation (100 %) in remote areas. The bias was attributed to both data range shifts and altered feature-target relationships (e.g., NH₃ emission vs. Dwet-NH₄⁺). Additionally, inconsistencies among multi-source datasets and limitations of ML structure further introduced uncertainties. This study quantified previously unaddressed input data-induced uncertainties in ML-based Nr deposition research, providing critical insights for reliable application of ML-derived data in Nr management. The proposed uncertainty assessment framework is also applicable to other ML-based geospatial interpolation tasks facing data scarcity challenges.
- Preprint
(3542 KB) - Metadata XML
-
Supplement
(2572 KB) - BibTeX
- EndNote
Status: final response (author comments only)
- RC1: 'Comment on egusphere-2025-6160', Anonymous Referee #1, 28 Jun 2026
-
RC2: 'Comment on egusphere-2025-6160', Anonymous Referee #2, 11 Jul 2026
This manuscript tackles a real and under-quantified problem—input-data-induced dataset shift in ML-based reactive nitrogen wet deposition—and the three-case sensitivity design (sample size, spatial distribution, site-type imbalance) is well-conceived and clearly executed. The findings are useful for the community and the proposed framework has broader applicability. However, several issues need to be resolved before acceptance.
- Lines 148–150: The regional deposition estimates in Section 3.4 and the China-average values in Section 3.1 are reported for 2000–2020, and the emission features are stated to cover 2000–2020. However, the observational targets end in 2018 (EANET 2000–2018; NADMN 2010; NNDMN 2010–2015), and the authors’ own cited HTAP_v3 reference (Crippa et al., 2023) explicitly covers 2000–2018. Please reconcile how predictions and emission inputs were extended to 2019–2020, or correct the stated period to 2000–2018 throughout.
- Lines 126–135 and 408–413: The reconstruction of Dwet from the predicted Cwet is underspecified, which undermines the deposition-amount results. The authors predict concentration (Cwet) rather than deposition to avoid precipitation-related uncertainty, yet the headline deposition estimates and sensitivity impacts in Section 3.4 (Figs. 9–10) are expressed as Dwet. Since Eq. (1) implies Dwet = Cwet × Prep, the deposition estimates must reintroduce a precipitation product. The manuscript does not state which precipitation dataset was used for this back-conversion, nor whether its uncertainty is propagated into the reported Dwet deviations. This is important because the paper’s central deliverable is Dwet uncertainty. Please clarify the conversion procedure and its treatment of precipitation uncertainty.
- Lines 159–165 versus Lines 502–511: In Section 2.2 the authors justify using emission data by reporting a "minimal difference" between emission- and satellite-based models for 2008–2010. However, in Section 4.1 they show that replacing NH3 emission with satellite NH3 VCD in the S3-Remote scenario raises R from 0.36 to 0.58 and improves the emission–Cwet slope agreement. Since remote-site behavior is central to the paper’s conclusions, the aggregate "minimal difference" finding may not hold in exactly the regime that matters most. I recommend qualifying the Section 2.2 statement to note that the equivalence is a domain-average result and does not extend to remote sites.
- Line 358: Section-numbering error. The subsection "3.2.3 Influence from imbalanced number of site types" follows "3.3.1 Influence from spatial distribution of measurements". Given the parent heading 3.3, the site-type subsection should be numbered 3.3.2, not 3.2.3.
- Lines 274–275 versus Lines 147–156, 164: Unreconciled feature count. The SFR calculation states "13,000 samples with 16 features," but the described feature set—2 emission precursors (NOx and NH3), 8 meteorological variables, and 2 geographic variables—sums to 12. Please reconcile the feature count (e.g., explicitly list all 16 features), since SFR is a stated basis for the sample-size conclusions.
- Line 226 (Equation 2): Typesetting error in the correlation coefficient formula. As printed, the denominator is a difference of two square-root terms, and the second term appears as (Yi − Y)2 rather than (Yi − Ȳ)2. The Pearson correlation denominator should be the product of the two root-sum-of-squares terms with mean-centered Y. Please correct the equation.
- Line 262 versus Line 410, plus editorial errors: Numeric clarification and typographical cleanup. The China-average Dwet-NH4+ is given as 4.8 kg N ha-1 yr-1 while the Base-case domain-average Dwet-NH4+ is ~9.0 kg N ha-1 yr-1 (Line 410); please state explicitly that these refer to China versus the full EA+SEA domain so readers do not read them as contradictory. In addition, please correct recurring typographical errors, including "Dwelt" (Line 21), "ML-base" (Line 63), "SWand" (Lines 321, 333), and "The the NADMN" (Line 600). A careful language edit throughout is advisable.
Citation: https://doi.org/10.5194/egusphere-2025-6160-RC2
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 693 | 406 | 81 | 1,180 | 160 | 82 | 115 |
- HTML: 693
- PDF: 406
- XML: 81
- Total: 1,180
- Supplement: 160
- BibTeX: 82
- EndNote: 115
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
This manuscript addresses an important issue: how input-data-induced dataset shift affects ML-based estimates of reactive nitrogen wet deposition. The topic is relevant and the case study is potentially useful. However, I recommend reconsideration after major revisions.
The main concerns are: (1) the contribution to GMD needs to be more clearly framed as an assessment framework/case study rather than a new modelling system; (2) reproducibility is not yet sufficient, as the archived notebooks lack a complete executable workflow, environment information, and full figure-generation scripts; (3) the validation strategy based on random train/test splits and random 10-fold CV may overestimate generalization because the data have station-level, temporal, and seasonal dependence; (4) the attribution of S2/S3 biases to dataset shift is partly confounded by data source, region, site type, and time coverage; and (5) the 6000-9000 sample threshold and the Dwet uncertainty statements should be presented more cautiously.
I suggest that the authors improve the reproducibility package, clarify the feature set and code-method consistency, add grouped or spatial/temporal validation tests, and narrow the conclusions accordingly.