In semi-arid regions, the timing and duration of the rainy season determine plant water availability, which directly impacts food security. Rainy season metrics, which aim to define and, in some cases, predict the onset and end of rains can support agricultural planning, such as scheduling planting dates and managing water resources. However, these metrics based on precipitation time series do not always accurately reflect plant water availability, and the variety of available metrics can complicate the selection of the most suitable one. This study demonstrates that rainy season metrics are more useful for agricultural purposes when their parameters are calibrated using local vegetation data. Furthermore, a metric's ability to capture observed vegetation variability can indicate its applicability over larger spatial or temporal scales. We test this hypothesis in the semi-arid Rio Santa basin in the Peruvian Andes by evaluating seven common rainy season metrics, both calibrated and uncalibrated, against land surface phenology data obtained from 18 years of satellite-derived Normalised Difference Vegetation Index (NDVI) data. Additionally, we introduce a new bucket-type metric that incorporates a simplified water balance, considering both accumulation and storage. To test the robustness of the metrics under future climate scenarios, we examine the sensitivity of these metrics to variations in rainfall intensity and frequency using statistically downscaled CMIP5 rainfall data for historical (1981–2018) and future (2019–2100) periods under RCP 4.5 and 8.5 scenarios. Our results show that calibrating metrics using vegetation data improves their consistency in capturing the start and end dates of the rainy season. The newly introduced bucket metric outperforms the other metrics in both accuracy and robustness. However, some established metrics exhibit sensitivities that raise concerns about their applicability under potential shifts in rainfall patterns due to climate change. Overall, CMIP5 projections reveal no consistent trends in rainy season onset and only a slight delay in rainy season end, with inter-annual variability and ensemble spread being the dominant factors. Our findings highlight the importance of calibrating metrics and stress-testing them across various climate conditions to ensure their agricultural relevance. The framework introduced here can easily be adapted for application in other semi-arid regions.