Rethinking Machine Learning Weather Normalisation: A Refined Strategy for Short-term Air Pollution Policies

Abstract. Air pollution causes millions of premature deaths annually, driving widespread implementation of clean air interventions. Quantitative evaluation of the efficacy of such interventions is critical in air quality management. Machine learning-based weather normalization (ML-WN) has been employed to isolate meteorological influences from emission-drive changes; however, it has its own limitations, particularly when abrupt emission shifts occur, e.g., after an intervention. Here we developed a logical evaluation framework, based on paired observational datasets and a test of ‘ML algebra’ (i.e., the ‘commutation’ of a normalisation step), to show that ML-WN significantly underestimates the immediate effects of short-term interventions on NO_X, with discrepancies reaching up to 42 % for one-week interventions. This finding challenges assumptions about the robustness of ML-WN for evaluating short-term policies, such as emergency traffic controls or episodic pollution events. We propose a refined approach (MacLeWN) that explicitly accounts for intervention timing, reducing underestimation biases by >90 % in idealised but plausible cases studies. We applied both approaches to evaluate the impact of COVID-19 lockdown on NO_X as measured at Marylebone Road, London. For the one-week period after the lockdown, ML-WN estimates approximately 17 % smaller NO_X reductions compared to MacLeWN, and such underestimation diminishes as policy duration extends, decreasing to ~10 % for one-month and becoming insignificant after three months. Our findings indicate the importance of carefully selecting evaluation methodologies for air quality interventions, suggesting that ML-WN should be complemented or adjusted when assessing short-term policies. Increasing model interpretability is also crucial for generating trustworthy assessments and improving policy evaluations.