the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 18 Aug 2023
Technical Note: Accurate, reliable and high resolution air quality predictions by improving the Copernicus Atmosphere Monitoring Service using machine learning techniques
Abstract. Starting from the regional air quality forecasts produced by the Copernicus Atmosphere Monitoring Service (CAMS), we propose a novel post-processing approach to improve and downscale results on a finer scale. Our approach is based on the combination of Ensemble Model Output Statistics (EMOS) with a spatio-temporal interpolation process performed through the Stochastic Partial Differential Equation-Integrated Nested Laplace Approximation (SPDE-INLA). Our interpolation approach includes several spatial and spatio-temporal predictors, including meteorological variables. A use-case is provided, scaling down the CAMS forecasts on the Italian peninsula. The calibration is focused on the concentrations of several air quality pollutants (PM10, PM2.5, NO2 and O3) at daily resolution from a set of 750 monitoring sites, distributed throughout the Italian country. Our results show the key role played by conditioning variables to improve the forecast capabilities of ensemble predictions, thus allowing a net improvement of the calibration with respect to ordinary EMOS strategies. From a deterministic point of view, the predictive model performance shows a significant improvement of the performance of the raw ensemble forecast, with an almost zero bias, significantly reduced root mean square errors and correlations almost always higher than 0.9 for each pollutant; moreover, the post-processing approach is able to significantly improve the prediction of exceedances, even for very low thresholds, such as those recently recommended by the World Health Organisation. This is particularly significant if a forecasting approach is to be used to predict air quality conditions and plan adequate human health protection measures, even for low alert thresholds. From a probabilistic point of view, the forecast quality was verified in terms of reliability and credible intervals. After post-processing, the predictive probability density functions were sharp, and much better calibrated than the raw ensemble forecast. Finally, we present some additional outcomes based on a set of gridded (4 km x 4 km) daily maps covering the whole Italian country, for the detection of areas where pollution peaks forecasts (exceedances of the regulatory thresholds) occur.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(2304 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2304 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2023-1015', Anonymous Referee #2, 27 Aug 2023
The authors present a method to downscale the modeled atmospheric compositions over Italy. The ensemble of CMAS model simulations was calibrated based on multi-model regressions with observations; the calibrated concentrations were then downscaled using geographical, meteorological variables and atmospheric observations. The topic is interesting, and the method is helpful for better prediction of atmospheric composition. However, there are still issues that need to be addressed before the paper can be considered for publication.
Comments:
- The title of this manuscript emphasizes the application of machine learning. However, the methods described in this manuscript, such as calibration of the ensemble in the first stage and statistical modelling of the space-time process in the second stage, appear to be statistical methods instead of machine learning.
- calibration of the ensemble: here the objective is to produce calibrated concentrations based on the linear regressions of multi-models and observations with parameters b1, … bm. I am a little surprised to see that this process needs to be repeated every day. I am wondering whether there are temporal and spatial changes in the parameters b1, … bm in the model training and what these changes in the parameters b1, … bm represent.
- statistical modelling of the space-time process: similar to the above question, does this process need to be repeated every day?
- Section 4.2-4.3: I suggest shortening these two sections and perhaps moving some figures into supplement because the parameters in both Stage 1 and Stage 2 are trained with observations, and it is thus expected to see some improvements after these two Stages.
- Section 4.4: In contrast, it could be better to extend this section as it is most interesting to the readers. For example, Fig. 7 is not convincing as these three stations may not provide a good representation of the whole domain. It could be better to provide scatter plots to show the overall performances of the predictions; While the high-resolution PM concentrations in Fig. 8 are interesting, it is useful to show the map of the differences between predictions and observations to demonstrate the spatial performance of the predictions; in addition, as the model needs to be trained every day, I am wondering whether the performance of the predictions has seasonal variabilities.
- 7: is it possible that this figure overestimates PM concentrations over rural areas because most stations in Table 1 are higher polluted urban and suburban stations?
Technical Comments:
- A flow chart is suggested to provide a clearer description of the methods.
- It would be helpful to provide a List to show the variables which were used in the model training including their temporal and spatial resolutions.
- Why the Section number of Stage 1 is 3.1.1 but 3.2 for Stage 2? I assume the description of these two stages is parallel.
- Lines 130-131: why most met predictors are selected at 12 UTC?
Citation: https://doi.org/10.5194/egusphere-2023-1015-RC1 - AC1: 'Reply on RC1', Angelo Riccio, 11 Nov 2023
-
RC2: 'Comment on egusphere-2023-1015', Anonymous Referee #1, 04 Sep 2023
This paper describes a statistical calibration and downscaling method to improve ensemble 1-day forecast of air quality over Italy. The two-step post-processing is evaluated comprehensively, focusing on the ability to capture air pollution exceedance. As a "Technical Note" paper, the proposed method and evaluation metrics are insightful for the community of air quality modeling and forecasting. Some details and methodology sensitivity evaluations are missing. I support the publication of this paper on ACP, if the following comments can be addressed:
1) Section 2.2: It is suggested to provide a table of all the data and their sources in the supplement or other appropriate locations of the paper. Too many URLs are present in this section.
2) Line 132: Temperature is a very important parameter relevant with photochemistry and lifetime of air pollutants. Why is it not included in the predictor list?
3) Line 200: Since the sites are unevenly distributed and many non-urban areas are not monitored. I would anticipate significantly reduced site density in Central and South Italy. So should we also consider more even inclusion/representation of sites in different part of Italy?
4) Section 4.1: I think it is well anticipated that the 11 models will have varying biases and precision. Maybe this section can be moved to the supplement?
5) Table 2: For all the four pollutants and in the "training" and "prediction" rows, the absolute biases are amplified from Step 1 to Step 2. It appears unusual and not found in previous studies. Why and does it matter?
6) If Figure 2 is only briefly discussed and Table 2 is mainly used in Section 4.2.1, maybe Figure 2 should also be moved to the supplement?
7) Section 4.4: NO2 has the strongest spatiotemporal variability due to its short lifetime. I believe case studies using NO2 can provide the most relevant information about model capability. Why is PM10 discussed here? Should similar results for the other pollutants be included in the supplement?
8) Figure 8: Please 1) add a map of median of raw predictions and 2) add observed values on the maps. Also, how to assess if the predicted values over unmonitored areas are accurate? Line 346 discussed "extrapolation ability", but quantitative evaluation of such ability is missing. Some "spatial-clustered" cross-validation idea (e.g., doi: s41467-020-18321-y) might be useful.
Citation: https://doi.org/10.5194/egusphere-2023-1015-RC2 - AC3: 'Reply on RC2', Angelo Riccio, 11 Nov 2023
-
RC3: 'Comment on egusphere-2023-1015', Anonymous Referee #3, 14 Sep 2023
This paper developed a statistical two stage method to better forecast the pollutant levels. The methods are evaluated with respect to key statistical metrics of both deterministic and probabilistic nature. The idea presented in this work is interesting and is of practical importance. The paper overall has good technical quality, although improvements can be made to further improve the manuscript. I suggest publication of this work after the following comments are addressed.
Major:
- Line 173: Why are three days’ data used to train the coefficients in the first stage? Are the coefficients sensitive to the number of days used for training.
- Table 2: From this table it seems that stage 2 worsens the prediction in terms of bias as well the RMSE of PM2.5. What is the reason for this?
- Line 349: More technical description can be provided, e.g., how are the coefficients used in stage 1 obtained? Are they the same as those trained in previous sections? The application in 4.4 is quite interesting and this section could be expanded to include more details.
- Figure 8 and related text: Please provide comparison with observations.
- I’d like to see some comments on the computational cost of the current method. Low computational cost indicates sensitivity studies (e.g., with respect to spatiotemporal predictors) can be easily performed to potentially improve the current method.
Minor:
- Line 134: Add references for the relation between temperature, wind speed, RH and ozone, PM and NO2.
Technical:
- Line 179: ‘given in Appendix A’.
- Can the authors also add legends to Figure 7 instead of only describing them in the text?
Citation: https://doi.org/10.5194/egusphere-2023-1015-RC3 - AC2: 'Reply on RC3', Angelo Riccio, 11 Nov 2023
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2023-1015', Anonymous Referee #2, 27 Aug 2023
The authors present a method to downscale the modeled atmospheric compositions over Italy. The ensemble of CMAS model simulations was calibrated based on multi-model regressions with observations; the calibrated concentrations were then downscaled using geographical, meteorological variables and atmospheric observations. The topic is interesting, and the method is helpful for better prediction of atmospheric composition. However, there are still issues that need to be addressed before the paper can be considered for publication.
Comments:
- The title of this manuscript emphasizes the application of machine learning. However, the methods described in this manuscript, such as calibration of the ensemble in the first stage and statistical modelling of the space-time process in the second stage, appear to be statistical methods instead of machine learning.
- calibration of the ensemble: here the objective is to produce calibrated concentrations based on the linear regressions of multi-models and observations with parameters b1, … bm. I am a little surprised to see that this process needs to be repeated every day. I am wondering whether there are temporal and spatial changes in the parameters b1, … bm in the model training and what these changes in the parameters b1, … bm represent.
- statistical modelling of the space-time process: similar to the above question, does this process need to be repeated every day?
- Section 4.2-4.3: I suggest shortening these two sections and perhaps moving some figures into supplement because the parameters in both Stage 1 and Stage 2 are trained with observations, and it is thus expected to see some improvements after these two Stages.
- Section 4.4: In contrast, it could be better to extend this section as it is most interesting to the readers. For example, Fig. 7 is not convincing as these three stations may not provide a good representation of the whole domain. It could be better to provide scatter plots to show the overall performances of the predictions; While the high-resolution PM concentrations in Fig. 8 are interesting, it is useful to show the map of the differences between predictions and observations to demonstrate the spatial performance of the predictions; in addition, as the model needs to be trained every day, I am wondering whether the performance of the predictions has seasonal variabilities.
- 7: is it possible that this figure overestimates PM concentrations over rural areas because most stations in Table 1 are higher polluted urban and suburban stations?
Technical Comments:
- A flow chart is suggested to provide a clearer description of the methods.
- It would be helpful to provide a List to show the variables which were used in the model training including their temporal and spatial resolutions.
- Why the Section number of Stage 1 is 3.1.1 but 3.2 for Stage 2? I assume the description of these two stages is parallel.
- Lines 130-131: why most met predictors are selected at 12 UTC?
Citation: https://doi.org/10.5194/egusphere-2023-1015-RC1 - AC1: 'Reply on RC1', Angelo Riccio, 11 Nov 2023
-
RC2: 'Comment on egusphere-2023-1015', Anonymous Referee #1, 04 Sep 2023
This paper describes a statistical calibration and downscaling method to improve ensemble 1-day forecast of air quality over Italy. The two-step post-processing is evaluated comprehensively, focusing on the ability to capture air pollution exceedance. As a "Technical Note" paper, the proposed method and evaluation metrics are insightful for the community of air quality modeling and forecasting. Some details and methodology sensitivity evaluations are missing. I support the publication of this paper on ACP, if the following comments can be addressed:
1) Section 2.2: It is suggested to provide a table of all the data and their sources in the supplement or other appropriate locations of the paper. Too many URLs are present in this section.
2) Line 132: Temperature is a very important parameter relevant with photochemistry and lifetime of air pollutants. Why is it not included in the predictor list?
3) Line 200: Since the sites are unevenly distributed and many non-urban areas are not monitored. I would anticipate significantly reduced site density in Central and South Italy. So should we also consider more even inclusion/representation of sites in different part of Italy?
4) Section 4.1: I think it is well anticipated that the 11 models will have varying biases and precision. Maybe this section can be moved to the supplement?
5) Table 2: For all the four pollutants and in the "training" and "prediction" rows, the absolute biases are amplified from Step 1 to Step 2. It appears unusual and not found in previous studies. Why and does it matter?
6) If Figure 2 is only briefly discussed and Table 2 is mainly used in Section 4.2.1, maybe Figure 2 should also be moved to the supplement?
7) Section 4.4: NO2 has the strongest spatiotemporal variability due to its short lifetime. I believe case studies using NO2 can provide the most relevant information about model capability. Why is PM10 discussed here? Should similar results for the other pollutants be included in the supplement?
8) Figure 8: Please 1) add a map of median of raw predictions and 2) add observed values on the maps. Also, how to assess if the predicted values over unmonitored areas are accurate? Line 346 discussed "extrapolation ability", but quantitative evaluation of such ability is missing. Some "spatial-clustered" cross-validation idea (e.g., doi: s41467-020-18321-y) might be useful.
Citation: https://doi.org/10.5194/egusphere-2023-1015-RC2 - AC3: 'Reply on RC2', Angelo Riccio, 11 Nov 2023
-
RC3: 'Comment on egusphere-2023-1015', Anonymous Referee #3, 14 Sep 2023
This paper developed a statistical two stage method to better forecast the pollutant levels. The methods are evaluated with respect to key statistical metrics of both deterministic and probabilistic nature. The idea presented in this work is interesting and is of practical importance. The paper overall has good technical quality, although improvements can be made to further improve the manuscript. I suggest publication of this work after the following comments are addressed.
Major:
- Line 173: Why are three days’ data used to train the coefficients in the first stage? Are the coefficients sensitive to the number of days used for training.
- Table 2: From this table it seems that stage 2 worsens the prediction in terms of bias as well the RMSE of PM2.5. What is the reason for this?
- Line 349: More technical description can be provided, e.g., how are the coefficients used in stage 1 obtained? Are they the same as those trained in previous sections? The application in 4.4 is quite interesting and this section could be expanded to include more details.
- Figure 8 and related text: Please provide comparison with observations.
- I’d like to see some comments on the computational cost of the current method. Low computational cost indicates sensitivity studies (e.g., with respect to spatiotemporal predictors) can be easily performed to potentially improve the current method.
Minor:
- Line 134: Add references for the relation between temperature, wind speed, RH and ozone, PM and NO2.
Technical:
- Line 179: ‘given in Appendix A’.
- Can the authors also add legends to Figure 7 instead of only describing them in the text?
Citation: https://doi.org/10.5194/egusphere-2023-1015-RC3 - AC2: 'Reply on RC3', Angelo Riccio, 11 Nov 2023
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 345 | 117 | 24 | 486 | 10 | 11 |
- HTML: 345
- PDF: 117
- XML: 24
- Total: 486
- BibTeX: 10
- EndNote: 11
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Cited
Angelo Riccio
Elena Chianese
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2304 KB) - Metadata XML