Quantifying the responses of AI precipitation forecast errors to reanalysis errors using analysis of covariance

Abstract. AI weather models are commonly trained by climate reanalysis datasets that inevitably influence AI weather forecasts. Given the importance of precipitation, it is essential to quantify how AI forecast errors respond to reanalysis errors. This paper proposes the use of the analysis of covariance (ANCOVA) method to diagnose the responses. Specifically, extreme precipitation events are identified from historical observations; both forecast and reanalysis errors are calculated against observations during the evaluation period; and the ANCOVA method is fitted to separate the responses to reanalysis errors from changes in response coefficients and additional offsets associated with extreme precipitation. A case study is devised for GraphCast forecasts and European Centre for Medium-range Weather Forecasts (ECMWF) Reanalysis v5 (ERA5), using Daymet observations across 1426 catchments in the Catchment Attributes and MEteorology for Large-Sample SPATially distributed analysis (CAMELS-SPAT) dataset. The results show that the responses of GraphCast forecast errors to ERA5 reanalysis errors weaken progressively with increasing lead time. Under non-extreme conditions, the median response coefficient declines from 0.70 at the 1d lead time to 0.27 at 9d, indicating that each 1 mm/d increase in ERA5 precipitation errors leads to a median increase of 0.70 and 0.27 mm/d in GraphCast precipitation errors, respectively. Extreme precipitation amplifies the underestimation by GraphCast forecasts, with the median additional offset across 1426 catchments decreasing from −1.75 to −10.43 mm/d when the lead time increases from 1d to 9d. Overall, the ANCOVA method contributes to understandings of factors governing precipitation forecast errors and highlights the need to consider inherent uncertainties in climate reanalysis when training and evaluating AI weather models.