Score-based Filtering for Land Data Assimilation

Abstract. Land data assimilation (DA) improves estimates of land states by integrating observations with land surface model (LSM) simulations. Ensemble Kalman filters (EnKFs) are widely used but assume Gaussian forecast-error distributions, limiting their ability to represent non-Gaussian errors under nonlinear dynamics. Score-based filters (SFs) relax this assumption by using diffusion models to estimate the score function of non-Gaussian forecast distribution and assimilating observations through an approximate likelihood score that guide reverse diffusion process to generate analysis samples, providing a flexible non-Gaussian DA framework that has shown promise in idealized experiments. However, their application to land DA with complex LSMs remains limited by forecast-score estimation errors at computationally affordable ensemble sizes and empirically calibrated likelihood-score approximations that can be sensitive to noisy satellite retrievals. To address these limitations, we develop LandSF which uses stochastic differential equation editing for reverse-diffusion initialization with reduced sensitivity to forecast-score errors, and moment-matching posterior sampling to explicitly approximate likelihood-score without empirical calibration. We evaluate LandSF against the ensemble transform Kalman filter (ETKF) and a representative SF using a Kolmogorov-flow experiment and a realistic land DA experiment that assimilates SMAP soil moisture retrievals into the Common Land Model (CoLM). In the Kolmogorov-flow experiment, both SFs maintain low analysis errors as the forecast-error distribution evolves from Gaussian to non-Gaussian, whereas ETKF deteriorates rapidly. Compared with the benchmark SF, LandSF reduces the average RMSE by about 58 % and substantially suppresses spatial error artifacts. These advantages carry over to realistic land DA, where LandSF reduces ubRMSE relative to the benchmark SF at 64 % of the independent validation stations, with a mean reduction of about 2.28 %. LandSF also consistently improves both ubRMSE and R over open-loop simulation across all three station networks, while the benchmark SF is less robust across networks. Diagnostic experiments further attribute these improvements to the two methodological advances, which better preserve forecast-distribution statistics and reduce sensitivity to observational noise. These results demonstrate that LandSF provides a practical framework for realistic land DA applications with complex process-based LSMs.