| 04 Sep 2026
Learning from model failure: insights from an AI/SHAP-based temporal error decomposition of tracer-aided ecohydrological modelling
Abstract. Process-based models (PBM) have served as testing grounds for hypotheses, being falsified and refined through model evaluation based on temporally complex PBM errors. Yet, conventional evaluation practices (e.g., performance metrics, visual inspection across time series, etc.) largely depend on the user’s a priori knowledge to trace PBM errors to environmental forcings and are limited in resolving temporal error implications.
In this study, temporal characteristics of PBM errors were explored using a data-driven Artificial Intelligence (AI) model focusing on delayed and non-stationary signatures and linking them to the predictors (i.e., PBM input data). First, an ecohydrological isotope-enabled PBM, EcoHydroPlot, was calibrated against high-resolution datasets of water amounts and water stable isotopes (δ2H) in soil of two depth layers and tree xylem, monitored from June to October 2020 in a riparian willow plot in Berlin, Germany. Then, PBM errors were calculated for six calibration targets. Second, an ensemble of LSTMs (i.e., AI error analyser) was trained to reproduce the PBM error, which was then decomposed with SHapley Additive exPlanations (SHAP) across both the retrospective lag and the study period, yielding a two-dimensional (lag × time step) quantification of temporal attribution for each predictor.
The analyser reproduced 60–99% of the error variance, showing that these errors were not random noise but carried a systematic, learnable structure. The temporal characteristics of PBM errors depended on both predictors and targets, showing a general tendency that errors contributed by LAI, or targeting xylem δ2H, were attributed to earlier time lags, whereas those related to precipitation or sapflow were to recent lags. Over the study period, error propagations were observed in distinct patterns, i.e., Event-driven, Period-driven, and regime change signal, especially, near the transition period from growing into non-growing season.
The results showed that the AI error analyser could support evaluation of a PBM as a complementary tool, and as an example, its diagnoses were translated into hypotheses for a better representation of xylem δ²H in the PBM.
Review of the manuscript written by Jung et al. on „Learning from model failure: insights from an AI/SHAP-based temporal error decomposition of tracer-aided ecohydrological modelling”
This manuscript presents a methodological approach to analyse the simulation errors of the ecohydrological model EcoHydroPlot using LSTM models. The main goal is to improve the understanding of model errors and in particular their temporal occurrence and to explain them by AI models.
The manuscript addresses the current challenge of how to benefit from the two difference model directions (PBM, ML) and to generate new knowledge by implementing AI-approaches into the analyses of PBM.
However, I see potential to improve the manuscript to increase its potential.
Major comments:
1) My major point is that the readability of the manuscript could be improved. The sentence structures are often very complex and hard to understand. Several parts of the text include a too much information and too many relationships that are only understandable after reading the text several times. Examples for complex parts are the lines: 336-339, 379–381, 451-453, 532-535 as well as the chapter 3.2.2. I highly recommend checking the entire manuscript for complex sentences / paragraphs.
2a) The interpretation of the results is highly focused on differentiating between celerity and velocity targets. This subdivision of the targets should be included in the methodological section. An explanation why this subdivision is helpful and why which target belongs to celerity and velocity targets is needed.
2b) Moreover, a better explanation and interpretation in the results is required. To give an example: The content of the sentence in L. 340-341 is only understandable with more information.
3) I am not convinced that the use of the word “target” is optimal.
3a) L. 20: “Calibration targets” is not exactly clear. I suggest to be here more precise such as “Calibration target variable”. Target is more technical. I prefer here a more hydrological view.
3b) L. 231: “Calibrated against six targets”. This is again very technical. It is better to say against hydrological variables / components used as target value of the calibration or similar expressions.
Further comments:
L. 13-15: I agree that temporal error analysis is important. However, there were already in past studies dealing with temporal variability in model errors.
L. 26-27: “earlier time lag” and “recent lag” is too unclear for an abstract without reading the manuscript in advance.
L. 44: The link to sensitivity analysis is not clear here.
L. 46: The motivation of analysis temporal errors is too short. I suggest to add more studies dealing with temporal error analysis such as the studies from Ehret et al. (2011, HESS) or as already used later in the text from Reusser et al. (2011, WRR)
L. 87-89: This sentence could be misunderstood. It is a real problem if patterns are found without considering physical laws.
L. 181: Could you explain how the MODIS data set with coarse resolution could be used for this small-scale analysis?
L. 213: Transpiration from soil layer and evaporation from tree canopy?
L. 214: Why possible evaporation and not potential?
L. 221: Could you give some information about the Craig-Gordon model to better understand this point?
L. 236: This sentence needs to be reorganized: The DREAM algorithm which was (is) an … was used for efficient ….
L. 240: It is unclear how this likelihood function was detected.
Figure 2: The figure is very confusing with crossing arrows. Please reduce the amount of crossing arrows.
L. 265: Adam optimiser is not defined.
L. 290-295: The use of lag centroid and lag concentration is of high relevance for the study. However, their definitions are hidden in the section 3.2.2. I suggest separate paragraphs for both to make clear that these notations are important.
L. 291-292: The question is not clear: Why does the feature’s contribution sit?
L. 297-298: This sentence needs more explanation.
L. 333: Feature importance and its role needs to be explained in the methodological section.
L. 514: Could you give more information to “LAI-based phenology threshold”
Technical remarks:
L. 19: in a soil
L. 53: in a model
L. 156: missing bracket (2006)
L. 490: deeper … in the soil. There is something missing here.