the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 10 Sep 2026
Score-based Filtering for Land Data Assimilation
Abstract. Land data assimilation (DA) improves estimates of land states by integrating observations with land surface model (LSM) simulations. Ensemble Kalman filters (EnKFs) are widely used but assume Gaussian forecast-error distributions, limiting their ability to represent non-Gaussian errors under nonlinear dynamics. Score-based filters (SFs) relax this assumption by using diffusion models to estimate the score function of non-Gaussian forecast distribution and assimilating observations through an approximate likelihood score that guide reverse diffusion process to generate analysis samples, providing a flexible non-Gaussian DA framework that has shown promise in idealized experiments. However, their application to land DA with complex LSMs remains limited by forecast-score estimation errors at computationally affordable ensemble sizes and empirically calibrated likelihood-score approximations that can be sensitive to noisy satellite retrievals. To address these limitations, we develop LandSF which uses stochastic differential equation editing for reverse-diffusion initialization with reduced sensitivity to forecast-score errors, and moment-matching posterior sampling to explicitly approximate likelihood-score without empirical calibration. We evaluate LandSF against the ensemble transform Kalman filter (ETKF) and a representative SF using a Kolmogorov-flow experiment and a realistic land DA experiment that assimilates SMAP soil moisture retrievals into the Common Land Model (CoLM). In the Kolmogorov-flow experiment, both SFs maintain low analysis errors as the forecast-error distribution evolves from Gaussian to non-Gaussian, whereas ETKF deteriorates rapidly. Compared with the benchmark SF, LandSF reduces the average RMSE by about 58 % and substantially suppresses spatial error artifacts. These advantages carry over to realistic land DA, where LandSF reduces ubRMSE relative to the benchmark SF at 64 % of the independent validation stations, with a mean reduction of about 2.28 %. LandSF also consistently improves both ubRMSE and R over open-loop simulation across all three station networks, while the benchmark SF is less robust across networks. Diagnostic experiments further attribute these improvements to the two methodological advances, which better preserve forecast-distribution statistics and reduce sensitivity to observational noise. These results demonstrate that LandSF provides a practical framework for realistic land DA applications with complex process-based LSMs.
- Preprint
(9401 KB) - Metadata XML
-
Supplement
(2008 KB) - BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on egusphere-2026-5358', Anonymous Referee #1, 12 Oct 2026
-
RC2: 'Reply on RC1', Anonymous Referee #2, 25 Oct 2026
GENERAL COMMENTS
This manuscript introduces LandSF, a score-based filter tailored for land data assimilation that couples with the sophisticated process-based land surface model CoLM. Within the analysis routine of existing score-based filters, this framework incorporates two key innovations: the first applies SDEdit-style initialization to the reverse diffusion process starting from partially perturbed forecast members, which lowers reliance on the accuracy of forecast-score estimates while remaining feasible for moderately sized ensembles; the second adopts moment-matching posterior sampling (MMPS) to estimate the diffusion-time likelihood score, eliminating the need for empirically tuned damping or guidance parameters. The performance of this new method is tested in comparison with the ETKF approach and a reference score-based filter (B24) using a Kolmogorov-flow test case, as well as in a real-world application where SMAP surface soil moisture observations are assimilated into CoLM across a North American study region, with results validated against measurements from 110 ISMN stations.
This work tackles a pressing and meaningful research topic. As diffusion-based non-Gaussian data assimilation advances at a fast pace, extending its use to process-based land surface models represents a valuable step forward, moving beyond the simplified, idealized scenarios that have been the primary testbed for score-based filters to date. The formal link established between this method and the EnKF (detailed in Appendix A) is mathematically sound, the two designed experiments offer complementary perspectives, and the targeted diagnostic tests that isolate performance gains from the SDEdit initialization mechanism and from MMPS respectively stand out as a notable strength of the study. Overall, the manuscript is clear and logically structured, and the decision to make the CoLM coupling publicly available alongside archived code and supporting data is highly praiseworthy.
However, several issues should be resolved before the conclusions are fully supported. I therefore recommend publication after minor revisions.
- Absence of an EnKF baseline in the real-world land data assimilation experiment. In the current land DA test, only the performance of LandSF, B24, and the open loop is compared. Given that the core motivation of this study is to address the suboptimal performance of Gaussian ensemble filtering under non-Gaussian error conditions, it is necessary to either provide ETKF results under the CoLM framework (with identical ensemble size and perturbation scheme), or offer a clear, well-reasoned explanation for why such results are not included. This is essential to evaluate whether LandSF's observed advantage over B24 also translates to a tangible advantage over a properly calibrated EnKF in practical applications. Additionally, please explicitly indicate whether localization was applied in the ETKF implementation for the Kolmogorov-flow experiment, and provide a brief discussion of how localization functions in the land data assimilation experiment.
- Limited magnitude of performance gains and need for enhanced statistical rigor. Relative to the B24 benchmark, LandSF delivers an average ubRMSE reduction of 2.28% across 64% of observation stations, while B24 itself only yields a 1.78% average improvement compared to the open loop. The practical value of these relatively small improvements merits candid, balanced discussion.
- The assertion of "no tunable parameters" (Lines 289–290). In practice, LandSF still requires multiple case-specific selections: the starting step for SDEdit, the maximum number of GMRES iterations, the count of DDIM steps, the stochasticity strength parameter, and the fine-tuning setup. The properly defensible statement is that LandSF eliminates the observation-dependent guidance parameter used in B24, and the relevant wording should be adjusted to reflect this accurately.
- Ensemble size and uncertainty assessment. The score network is fine-tunedeach cycle using only an ensemble of N=100 members. We recommend quantifying the model's sensitivity to changes in ensemblesize, and assessing the calibration of the generated analysis ensemble via metrics including the spread-to-skill ratio, rank histograms, or CRPS, rather than relying solely on ensemble-mean ubRMSE and correlation coefficient R. Without these additional evaluations, the asserted advantage of LandSF for representing flow-dependent uncertainty remains unvalidated in the land data assimilation experiment.
- Computational requirements and upscaling performance. While the manuscript presents LandSF as a"practical framework", it provides no data on actual runtimes, GPU usage duration, or memory demand. We ask that you share the costs associated with per-cycle fine-tuning, sampling runs, and the 100-member CoLM integration, as well as discuss how the approach scales when applied to global domains, finer resolutions, and multi-variable or multi-layer state vectors.
- Definition of "land DA" scope. This work only assimilates surface soil moisture. We recommend toning down the current framing or outlining the pathway toassimilatingroot-zone soil moisture, snow, and land surface temperature, enabling joint state-parameter estimation, and assessing cold-season performance.
Citation: https://doi.org/10.5194/egusphere-2026-5358-RC2
-
RC2: 'Reply on RC1', Anonymous Referee #2, 25 Oct 2026
- EC1: 'Comment on egusphere-2026-5358', Dalei Hao, 10 Nov 2026
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 516 | 240 | 68 | 824 | 106 | 72 | 71 |
- HTML: 516
- PDF: 240
- XML: 68
- Total: 824
- Supplement: 106
- BibTeX: 72
- EndNote: 71
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
This manuscript by Fan et al. develops LandSF, a score-based filtering framework for land data assimilation. Applying score-based data assimilation to a complex land surface model is potentially valuable. The authors test the method in both an idealized nonlinear system and a realistic CoLM application. I find the two proposed modifications intuitive: initializing reverse diffusion from perturbed forecast members may reduce the dependence on an imperfectly estimated score, and explicitly accounting for uncertainty in the likelihood approximation may avoid overreacting to noisy observations. Overall, I think method in the paper is promising. However, the evidence supports the limited conclusion that LandSF may be a more stable variant of the benchmark method B24 under the tested configurations. Demonstrating that LandSF is broadly practical and advantageous for land DA will require more transparent tuning, an ensemble-size study, computational-cost reporting, and a conventional DA baseline in the realistic application.
Specific comments:
As I understand it, the two methods use the same score-network architecture, training procedure, diffusion schedule, and sampling setup. The main differences are the SDEdit initialization and the MMPS likelihood approximation. The claim that LandSF has no tunable parameters is too strong. It avoids the additional B24 guidance parameter, but it still depends on important choices such as the SDEdit starting point, number of sampling steps, score-network configuration, training settings, and number of GMRES iterations.
The comparison with B24 also needs more transparency. The B24 guidance threshold is said to be optimized separately for each experimental setting, but the manuscript does not explain how it was selected, over what range, using what metric, or whether independent validation data were used. This matters because the paper itself shows that B24 is sensitive to this parameter. Please describe the full tuning procedure and show how B24 performs across a reasonable range of guidance settings in the land experiment. Otherwise, it is difficult to know whether LandSF is more robust or whether B24 was not optimally configured.
Another major issue is that the motivation of the paper is limited ensemble size, but the experiments use only one ensemble size, 100 members. The manuscript argues that LandSF is less sensitive to forecast-score error when the ensemble is computationally constrained, but it does not show how performance changes with ensemble size. It is therefore unclear whether LandSF can achieve similar skill with fewer members than B24 or ETKF, how small the ensemble can be before performance degrades, or whether 100 members are sufficient in this high-dimensional application. An ensemble-size sensitivity analysis would greatly strengthen the paper.
The paper presents LandSF as practical for complex land DA but does not report its computational cost. The method requires repeated score-network training or fine-tuning, number of reverse-diffusion sampling steps, and MMPS calculations involving automatic differentiation and GMRES solves. It may improve robustness, but it is not obvious whether it is affordable at higher resolution, with more frequent assimilation, or with a larger state vector. Please report total computational cost and ideally separate the cost of model propagation, score training, and analysis sampling. A scaling discussion such as how LandSF's cost and performance change as the problem gets bigger would also be useful.
I also think the realistic land experiment needs a conventional DA benchmark. ETKF is used in the Kolmogorov-flow experiment but not in the CoLM/SMAP experiment. Open loop is useful, but it is not a substitute for an EnKF/ETKF baseline. Without a standard land DA method in the realistic experiment, the paper demonstrates that LandSF can improve upon B24 under the selected setup, but not whether it improves upon a commonly used approach in realistic land DA. Including ETKF with the same ensemble would substantially improve the practical relevance of the study.
For the CoLM experiment, the reported LandSF improvement over B24 is modest, although the spatial robustness may still be meaningful. The claims should therefore be supported with stronger statistical analysis. The station results are paired, and the observations are spatially and temporally dependent, so simple confidence intervals based on independent samples may not be sufficient. A paired or block-bootstrap analysis would be more convincing. It would also help to include a direct LandSF-minus-open-loop panel in Fig. 5. The current figure clearly shows B24-to-LandSF differences, but it is harder to see where LandSF itself improves upon open loop and where it still degrades performance.
The experimental design also needs more detail. Please report the assimilation frequency, number of retained SMAP observations, and their spatial coverage after screening. More information on soil-layer discretization and the mapping from CoLM soil moisture to the SMAP observation space would also be helpful.
Line 9: explicitly state the “two methodological” advances so line 19 can refer to them
Line 16: ubRMSE not defined
Line 18: describe the three station networks
Define “MMPS” in caption of Figure 1
Equation 15 didn’t specify the map from the clean state to xt.
Lines 357-359: Four networks listed here, 3 in the abstract.
Figure 2: LandSF shows some localized large errors, such as the lower-right corner of column 4, please discuss
Figure 3: caption font size is too small
Figure 6 does not show LandSF performing better in every network. At SCAN and USCRN, ubRMSE is nearly identical for B24 and LandSF, and B24 has slightly higher R. So statement on line 411 should be softened. Why “ARM” is not compared by itself?
Lines 421 – 426. The authors describe LandSF's increments as 'smoother and reasonable' and attribute this to the explicit treatment of observation uncertainty by MMPS. Figure 8 suggests a different reading - LandSF's analysis increments appear to track the observation innovations less closely than B24's. A filter that consistently under-corrects toward the observations is not necessarily better, since shrinkage toward the forecast can produce smoothness without improving accuracy. The manuscript needs to show that the weaker correction reflects a justified balance between forecast and observation errors, not systematic under weighting of the observations.