Preprints
https://doi.org/10.5194/egusphere-2024-3703
https://doi.org/10.5194/egusphere-2024-3703
07 Jan 2025
 | 07 Jan 2025

Using Monte Carlo conformal prediction to evaluate the uncertainty of deep learning soil spectral models

Yin-Chung Huang, José Padarian, Budiman Minasny, and Alex B. McBratney

Abstract. Uncertainty quantification is a crucial step for the practical application of soil spectral models, particularly in supporting real-world decision making and risk assessment. While machine learning has made remarkable strides in predicting various physiochemical properties of soils using spectroscopy, predictions devoid of quantified uncertainty offer limited utility in guiding critical decisions. However, uncertainty quantification remains underutilised in the reporting of soil spectral models, with existing methods facing significant limitations. These approaches are either computationally demanding, fail to achieve the desired coverage of observed data, or struggle to handle out-of-domain uncertainty effectively. This study introduces the innovative use of Monte Carlo conformal prediction (MC-CP) as a novel approach to quantify uncertainty in the prediction of clay content from mid-infrared spectroscopy. We compared MC-CP with two established methods: (1) Monte Carlo dropout and (2) conformal prediction. Monte Carlo dropout generates prediction intervals for each sample and is effective at addressing larger uncertainties associated with out-of-domain data. However, it falls short in achieving the desired coverage – its 90 % prediction intervals only covered the observed values in 74 % of cases, well below the expected 90 % coverage. Conformal prediction, on the other hand, guarantees ideal coverage of true values but generates unnecessarily wide prediction intervals, making it overly conservative for many practical applications. In contrast, MC-CP successfully combines the strengths of both methods. It achieved a prediction interval coverage probability of 91 %, closely matching the expected 90 % coverage, and far surpassing the performance of Monte Carlo dropout. Additionally, the mean prediction interval width for MC-CP was 9.05 %, narrower than conformal prediction’s 11.11 %, while still effectively addressing the higher uncertainty in out-of-domain samples. By generating accurate prediction intervals alongside point predictions, MC-CP demonstrated its ability to deliver practical and reliable uncertainty quantification. This breakthrough enhances the real-world applicability of soil spectral models and represents a significant advancement in the field of soil science. The success of MC-CP paves the way for its integration into large-scale machine-learning models, such as soil inference systems, further revolutionising decision-making and risk assessment in soil science.

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors. Views expressed in the text are those of the authors and do not necessarily reflect the views of the publisher.
Share

Journal article(s) based on this preprint

22 Jul 2025
Using Monte Carlo conformal prediction to evaluate the uncertainty of deep-learning soil spectral models
Yin-Chung Huang, José Padarian, Budiman Minasny, and Alex B. McBratney
SOIL, 11, 553–563, https://doi.org/10.5194/soil-11-553-2025,https://doi.org/10.5194/soil-11-553-2025, 2025
Short summary
Yin-Chung Huang, José Padarian, Budiman Minasny, and Alex B. McBratney

Interactive discussion

Status: closed

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on egusphere-2024-3703', Anonymous Referee #1, 03 Mar 2025
    • RC3: 'Edit for RC1', Anonymous Referee #1, 14 Mar 2025
      • AC1: 'Reply on RC1', Yin-Chung Huang, 07 Apr 2025
    • AC1: 'Reply on RC1', Yin-Chung Huang, 07 Apr 2025
  • RC2: 'Comment on egusphere-2024-3703', Anonymous Referee #2, 14 Mar 2025
    • AC2: 'Reply on RC2', Yin-Chung Huang, 07 Apr 2025

Interactive discussion

Status: closed

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on egusphere-2024-3703', Anonymous Referee #1, 03 Mar 2025
    • RC3: 'Edit for RC1', Anonymous Referee #1, 14 Mar 2025
      • AC1: 'Reply on RC1', Yin-Chung Huang, 07 Apr 2025
    • AC1: 'Reply on RC1', Yin-Chung Huang, 07 Apr 2025
  • RC2: 'Comment on egusphere-2024-3703', Anonymous Referee #2, 14 Mar 2025
    • AC2: 'Reply on RC2', Yin-Chung Huang, 07 Apr 2025

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision | EF: Editorial file upload
ED: Reconsider after major revisions (further review by editor and referees) (09 Apr 2025) by Pedro Batista
AR by Yin-Chung Huang on behalf of the Authors (10 Apr 2025)  Author's response   Author's tracked changes   Manuscript 
ED: Referee Nomination & Report Request started (15 Apr 2025) by Pedro Batista
RR by Anonymous Referee #1 (22 Apr 2025)
RR by Anonymous Referee #2 (05 May 2025)
ED: Publish subject to technical corrections (06 May 2025) by Pedro Batista
ED: Publish subject to technical corrections (09 May 2025) by Peter Fiener (Executive editor)
AR by Yin-Chung Huang on behalf of the Authors (14 May 2025)  Author's response   Manuscript 

Journal article(s) based on this preprint

22 Jul 2025
Using Monte Carlo conformal prediction to evaluate the uncertainty of deep-learning soil spectral models
Yin-Chung Huang, José Padarian, Budiman Minasny, and Alex B. McBratney
SOIL, 11, 553–563, https://doi.org/10.5194/soil-11-553-2025,https://doi.org/10.5194/soil-11-553-2025, 2025
Short summary
Yin-Chung Huang, José Padarian, Budiman Minasny, and Alex B. McBratney
Yin-Chung Huang, José Padarian, Budiman Minasny, and Alex B. McBratney

Viewed

Total article views: 592 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
399 173 20 592 14 27
  • HTML: 399
  • PDF: 173
  • XML: 20
  • Total: 592
  • BibTeX: 14
  • EndNote: 27
Views and downloads (calculated since 07 Jan 2025)
Cumulative views and downloads (calculated since 07 Jan 2025)

Viewed (geographical distribution)

Total article views: 578 (including HTML, PDF, and XML) Thereof 578 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 10 Sep 2025
Download

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Short summary
Uncertainty quantification plays a crucial role in reporting machine learning models in soil spectroscopy. This study introduces Monte Carlo conformal prediction (MC-CP), a novel method for uncertainty quantification in deep learning soil spectral models. MC-CP outperformed two established methods, providing the most reliable results. Its efficiency and robustness make it a practical choice for implementing soil spectral models in decision-making.
Share