The large-scale mapping of plant biophysical and biochemical traits is essential for ecological and environmental applications. Given its finer spectral resolution and unprecedented data availability, hyperspectral data has emerged as a promising, non-destructive tool for accurately retrieving these traits. Machine and particularly deep learning models have shown strong potential in retrieving plant traits from hyperspectral data. However, when deploying these methods at large scales, reliably quantifying associated uncertainty remains a critical challenge, especially when models encounter out-of-domain (OOD) data, such as unseen geographic regions, species, biomes, or data acquisition modalities. Traditional uncertainty quantification methods for deep learning models, including deep ensembles (Ens_UN) and Monte Carlo dropout (MCdrop_UN), rely on the variance of predictions but often fail to capture uncertainty in OOD scenarios, leading to overoptimistic and potentially misleading uncertainty estimates. To address this limitation, we propose a distance-based uncertainty estimation method (Dis_UN) that quantifies prediction uncertainty by measuring dissimilarity in the predictor and embedding space between training and test data. Dis_UN leverages residuals as a proxy for uncertainty and employs dissimilarity indices in data manifolds to estimate worst-case errors via 95-quantile regression. We evaluate Dis_UN on a pre-trained deep learning model for prediction of multiple plant traits from hyperspectral images, analyzing its performance across OOD data, such as pixels containing spectral variation from urban surfaces, bare ground, water, clouds or open surface waters. For this study we target six leaf and canopy traits: Leaf mass per area (LMA), Chlorophyll (Chl), Carotenoids (Car), Nitrogen (N) content, Leaf area index (LAI) and Equivalent water thickness (EWT). Results indicate that Dis_UN effectively differentiates between OOD components and provides more reliable uncertainty estimates than traditional methods, which tend to underestimate the range of uncertainty (on average over traits 26.7 % for Ens_UN and 6.5 % for Dropout_UN). However, challenges remain for traits affected by spectral saturation. These findings highlight the advantages of distance-aware uncertainty quantification methods and underscore the necessity of diverse training datasets to minimize sampling biases and enhance model robustness. The proposed framework improves the reliability of uncertainty estimation in vegetation monitoring and offers a promising approach for broader applications.