A new set of indicators for model evaluation complementing to FAIRMODE’s MQO

Abstract. In this study, we assess the relevance and utility of several performance indicators developed within the FAIRMODE framework by evaluating eight CAMS models and their ensemble in calculating concentrations of key air pollutants, specifically NO2, PM2.5, PM10, and O3. The models' outputs were compared with observations that were not assimilated into the models. For NO2, the results highlight difficulties in accurately modelling concentrations at traffic stations, with improved performance when these stations are excluded. While all models meet the established criteria for PM2.5, indicators such as bias and Winter-Summer gradients reveal underlying issues in air quality modelling, questioning the stringency of the current criteria for PM2.5. For PM10, the combination of MQI, bias, and spatial-temporal gradient indicators prove most effective in identifying model weaknesses, suggesting possible areas of improvement. O3 evaluation shows that temporal correlation and seasonal gradients are useful in assessing model performance. Overall, the indicators provide valuable insights into model limitations, yet there is a need to reconsider the strictness of some indicators for certain pollutants.