Correction of Diurnal Errors in Dielectric-Based In-Situ Soil Moisture Measurements via a Stacked LSTM Framework

Abstract. Soil moisture (SM) is a fundamental variable in land-atmosphere interactions, yet in-situ measurements from dielectric-based sensors often suffer from systematic temperature sensitivity known as the Maxwell-Wagner polarization effect. This sensitivity induces spurious daytime peaks that contradict the physical reality of evapotranspiration-driven daytime dry-down. In this study, we propose a data-driven approach using stacked Long Short-Term Memory (LSTM) networks to correct these temperature-induced diurnal errors in the International Soil Moisture Network (ISMN) by integrating in-situ observations with physically constrained diurnal patterns from ERA5-Land and MERRA-2 reanalysis datasets.

The LSTM-based correction (ISMN_LSTM) effectively reverses the spurious positive correlations between diurnal cycles of SM and soil temperature to physically consistent negative correlations, demonstrating superior robustness across various sensor technologies compared to a recent reanalysis-informed Fourier filtering method that can struggle with regional biases. Performance evaluations reveal a significant enhancement in diurnal temporal correlation, with ISMN_LSTM achieving R=0.89 against reference observations compared to the raw ISMN (R=0.14). Gradient-based sensitivity analysis confirms that the model's predictive logic is rooted in physical processes, with short-lag (1–3 hours) thermal forcing from high-frequency components and broadly distributed sensitivity to low-frequency components representing background land surface state, reflecting a multi-scale information integration. Furthermore, the diurnally adjusted SM in flux tower observations reveals physically realistic land-atmosphere coupling with negative SM-latent heat flux correlations throughout the diurnal cycle, which are fundamentally misdiagnosed as weakly positive in the original observations. Overall, the LSTM-based correction framework provides a reliable foundation for the initialization of numerical weather prediction models, the validation of sub-daily satellite SM retrievals, and advancing the understanding of sub-daily land-atmosphere interactions.