Preprints
https://doi.org/10.5194/egusphere-2024-323
https://doi.org/10.5194/egusphere-2024-323
21 Feb 2024
 | 21 Feb 2024

Insights into the prediction uncertainty of machine-learning-based digital soil mapping through a local attribution approach

Jeremy Rohmer, Stephane Belbeze, and Dominique Guyonnet

Abstract. Machine learning (ML) models have become key ingredients for digital soil mapping. To improve the interpretability of their prediction, diagnostic tools have been developed like the widely used local attribution approach known as ‘SHAP’ (SHapley Additive exPlanation). However, the analysis of the prediction is only one part of the problem and there is an interest in getting deeper insights into the drivers of the prediction uncertainty as well, i.e. to explain why the ML model is confident, given the set of chosen covariates’ values (in addition to why the ML model delivered some particular results). We show in this study how to apply SHAP to the local prediction uncertainty estimates for a case of urban soil pollution, namely the presence of petroleum hydrocarbon in soil at Toulouse (France), which poses a health risk via vapour intrusion into buildings, direct soil ingestion or groundwater contamination. To alleviate the computational burden posed by the multiple covariates (typically >10) and by the large number of grid points on the map (typically over several 10,000s), we propose to rely on an approach that combines screening analysis (to filter out non-influential covariates) and grouping of dependent covariates by means of generic kernel-based dependence measures. Our results show evidence that the drivers of the prediction best estimate are not necessarily the ones that drive the confidence in these predictions, hence justifying that decisions regarding data collection and covariates’ characterisation as well as communication of the results should be made accordingly.

Jeremy Rohmer, Stephane Belbeze, and Dominique Guyonnet

Status: final response (author comments only)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on egusphere-2024-323', Anonymous Referee #1, 19 Mar 2024
  • RC2: 'Comment on egusphere-2024-323', Anonymous Referee #2, 11 Apr 2024
Jeremy Rohmer, Stephane Belbeze, and Dominique Guyonnet

Data sets

Data to run the synthetic test case Hannah Meyer https://github.com/HannaMeyer/CAST/tree/master/inst/extdata

Model code and software

R markdown - synthetic test case Jeremy Rohmer https://github.com/anrhouses/groupSHAP-uncertainty

Jeremy Rohmer, Stephane Belbeze, and Dominique Guyonnet

Viewed

Total article views: 228 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
179 40 9 228 5 3
  • HTML: 179
  • PDF: 40
  • XML: 9
  • Total: 228
  • BibTeX: 5
  • EndNote: 3
Views and downloads (calculated since 21 Feb 2024)
Cumulative views and downloads (calculated since 21 Feb 2024)

Viewed (geographical distribution)

Total article views: 229 (including HTML, PDF, and XML) Thereof 229 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 13 Apr 2024
Download
Short summary
Machine learning (ML) models have become key ingredients for digital soil mapping. To explain why the ML model is confident, we apply a popular method from the field of explainable artificial intelligence, i.e. based on the Shapley values, to the uncertainty prediction of hydrocarbon pollutants on an urban soil. To alleviate the implementation difficulties (number of factors, complex relationships between the factors, high resolution maps), a simple-but-efficient grouping approach is tested.