Beyond behavioural models: equifinality and overparameterisation undermine confidence in predictions by soil organic matter models
Abstract. The complexity of soil organic matter models is often not supported by sufficient data for parameter optimisation, resulting in the calibration of more parameters than can be reliably optimized with the available data. This results in equifinality, the phenomenon that multiple parameter sets generate behavioural models, i.e., similarly well-performing models that cannot be ruled out. As such trade-offs between model complexity and data availability are often overlooked for soil organic matter models, the aim of this study is to assess how equifinality affects the variability of predictions made by behavioural soil organic matter models. The results show that the number of identifiable parameters, those that do not compensate for one another, increases with the number of calibration constraints, but remained limited to five even under the most data-rich conditions. Furthermore, the size of particulate organic matter (POM) and mineral-associated organic matter (MAOM) can only be accurately simulated when data on these pool sizes are used, while the turnover rate of MAOM is reliably simulated only when Δ14C data for MAOM are provided. Regardless of the type of mathematical equations used (e.g., absolute vs. relative Michaelis-Menten kinetics), or the number of optimised parameters, the tested models were able to correctly reproduce the measurements in steady state. However, different model structures led to divergent predictions upon a doubling of organic matter inputs, while the variation in the response of the behavioural models was up to eight times larger for overparameterised models compared to models for which only identifiable parameters were optimised. Our results emphasise the necessity of optimising only identifiable model parameters to avoid hidden uncertainty in model predictions.
This manuscript discusses the issues of parameter identifiability and equifinality in the context of soil organic matter (SOM) decomposition models, and uses multiple model formulations and parameter optimizations to estimate variability across model structures and parameterizations and demonstrate how much predictive uncertainty may remain after models have been optimized using steady-state values.
I thought this was a very informative paper and a very useful study for highlighting challenges with modeling soil carbon cycle processes. The introduction explains the complex issues of parameter identifiability and related uncertainties clearly and makes a good case for paying more attention to these issues in soil models. The model simulations and approaches are well described and provide a clear demonstration of the key concepts of the paper, including how multiple model structures and parameter values can provide similar accuracy with respect to steady-state values while diverging when the steady state changes, and demonstrating how unidentifiable parameter pairs manifest in these types of models. Overall, the paper does a great job making an important argument and backs it up with strong modeling results.
I have a few specific comments:
Line 84-86: This is true for non-microbial models as well
Line 106: Is there a missing word after "decades"? Maybe "unless Δ14 data..."
Line 122: Typo in "assess"
Line 129: Provide a reference here for the DEzs algorithm)
Line 166: The SOM model actually does keep track of the DOM pool size
Line 182: The calibration procedure for the OC inputs was not explained clearly. If they were calibrated, does this make them essentially another parameter of the model? I think having a precise value for carbon inputs is actually quite optimistic for comparison with "real" field data, since estimating litter inputs (especially for root and root exudation fluxes) is very difficult
Line 296: Following on my previous comment, I think the normal situation with real measurements is actually worse than this, because you typically don't have accurate knowledge of total litter inputs. In a steady state system, knowing the inputs is equivalent to having soil heterotrophic respiration measurements, assuming there is no net leaching of DOM. And heterotrophic respiration is difficult to measure accurately if there are living roots. So, if anything this setup might be optimistic compared to a real study where carbon pool data is available.
Line 361: Could say explicitly here that the identifiable parameters were picked based on the identifiability analysis for each model structure, which is implied but wasn't clear until I looked at the supplementary tables
I had trouble figuring out how the values for the non-optimized parameters were picked, since they had to be fixed but the premise of the study is that the values are not well constrained. I guess the values come from the deterministic parameter calibration? I found that part a bit confusing
Line 455: Are the predictions more reliable? Or is the uncertainty underestimated? If the values of the non-optimized parameters are unknown but need to be fixed, this is adding a hidden uncertainty to the model (as mentioned in the Discussion), so I'm not sure it is actually more reliable. In Figure 5a and 5b, there is certainly a narrower distribution of predictions in the IPM approach, but this results from fixing the value of some unknown parameters. Couldn't this approach just as easily lead to a narrow but wrong result? So, perhaps 5b is a more accurate depiction of the actual predictive uncertainty in this situation where data is very limited, unless there are other constraints on the parameter values.
Line 461: This alone is INsufficient
Line 463: Did it reduce the uncertainty, or underestimate the uncertainty?
Section 4.2: I think this paragraph makes an important point very clearly
Line 521-522: I really like the "hidden uncertainty" phrasing here, which makes an important point about how to interpret these results
Line 533: Again, is it correct to say that uncertainty is reduced? I'm not sure this section is even making a point about reducing uncertainty. I think it mostly makes a strong argument that identifiability is an important analysis to do. I think part of the challenge here for the community is that identifiability analysis identifies a problem but does not really provide a solution for reducing uncertainty
Line 567: This is the big challenge, right? Because if the unidentifiable parameter values could be well constrained with observations or experiments they would not need to be optimized in the first place. In that sense, the identifiability issue is the beginning of the conversation, not the end of it