Preprints
https://doi.org/10.5194/egusphere-2024-4051
https://doi.org/10.5194/egusphere-2024-4051
04 Feb 2025
 | 04 Feb 2025
Status: this preprint is open for discussion and under review for Geoscientific Model Development (GMD).

Evaluating uncertainty and predictive performance of probabilistic models devised for grade estimation in a porphyry copper deposit

Raymond Leung, Alexander Lowe, and Arman Melkumyan

Abstract. Probabilistic models are used extensively in geoscience to describe random processes as they allow prediction uncertainties to be quantified in a principled way. These probabilistic predictions are valued in a variety of contexts ranging from geological and geotechnical investigations to understanding subsurface hydrostratigraphic properties and mineral distribution. However, there are no established protocols for evaluating the uncertainty and predictive performance of univariate probabilistic models, and few examples that researchers and practitioners can lean on. This paper aims to bridge this gap by developing a systematic approach that targets three objectives. First, geostatistics are used to check if the probabilistic predictions are reasonable given validation measurements. Second, image-based views of the statistics help facilitate large-scale simultaneous comparisons for a multitude of models across space and time, for instance, spanning multiple regions and inference periods. Third, variogram ratios are used to objectively measure the spatial fidelity of models. In this study, the model candidates include ordinary kriging and Gaussian Process, with and without sequential or correlated random field simulation. A key outcome are recommendations that encompass the FLAGSHIP statistics which examine the fidelity, likelihood, accuracy, goodness, synchronicity, histogram, interval tightness and precision of the model predictive distributions. These statistics are standardised, interpretable and amenable to significance testing. The proposed methods are demonstrated using extensive data from a real copper mine in a grade estimation task, and accompanied by an open-source implementation. The experiments are designed to emphasise data diversity and convey insights, such as the increased difficulty of future-bench prediction (extrapolation) relative to in-situ regression (interpolation). This work presents a holistic approach that enables modellers to evaluate the merits of competing models and employ models with greater confidence by assessing the robustness and validity of probabilistic predictions under challenging conditions.

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.
Share
Raymond Leung, Alexander Lowe, and Arman Melkumyan

Status: open (until 01 Apr 2025)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
Raymond Leung, Alexander Lowe, and Arman Melkumyan

Data sets

EUP3M: Evaluating uncertainty and predictive performance of probabilistic models—Python code for model construction and statistical analysis Raymond Leung and Alexander Lowe https://github.com/raymondleung8/eup3m/blob/main/data

Model code and software

EUP3M: Evaluating uncertainty and predictive performance of probabilistic models—Python code for model construction and statistical analysis Raymond Leung and Alexander Lowe https://doi.org/10.5281/zenodo.14533140

Raymond Leung, Alexander Lowe, and Arman Melkumyan

Viewed

Total article views: 36 (including HTML, PDF, and XML)
HTML PDF XML Total Supplement BibTeX EndNote
32 2 2 36 5 0 0
  • HTML: 32
  • PDF: 2
  • XML: 2
  • Total: 36
  • Supplement: 5
  • BibTeX: 0
  • EndNote: 0
Views and downloads (calculated since 04 Feb 2025)
Cumulative views and downloads (calculated since 04 Feb 2025)

Viewed (geographical distribution)

Total article views: 36 (including HTML, PDF, and XML) Thereof 36 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 06 Feb 2025
Download
Short summary
Probabilistic models are used in geoscience to enable quantification of uncertainties, particularly in predicting subsurface geochemical distributions. However, there is a lack of established protocols for evaluating the uncertainty and predictive performance of univariate probabilistic models. This paper presents a systematic approach which enables modellers to assess competing models, conduct large-scale model comparisons and validate probabilistic predictions using statistical measures.
Share