An optimized semi-empirical physical approach for satellite-based PM<sub>2.5</sub> retrieval: embedding machine learning to simulate complex physical parameters

Jin, Caiyi; Yuan, Qiangqiang; Li, Tongwen; Wang, Yuan; Zhang, Liangpei

doi:https://doi.org/10.5194/egusphere-2022-946

Preprints

https://doi.org/10.5194/egusphere-2022-946

Preprints

27 Oct 2022

| 27 Oct 2022

An optimized semi-empirical physical approach for satellite-based PM_2.5 retrieval: embedding machine learning to simulate complex physical parameters

Caiyi Jin, Qiangqiang Yuan, Tongwen Li, Yuan Wang, and Liangpei Zhang

Abstract. Satellite remote sensing of PM_2.5 mass concentration has become one of the most popular atmospheric research aspects, resulting in the development of different models. Among them, the semi-empirical physical approach constructs the transformation relationship between the aerosol optical depth (AOD) and PM_2.5 based on the optical properties of particles, which has strong physical significance. Also, it performs the PM_2.5 retrieval independently of the ground stations. However, due to the complex physical relationship, the physical parameters in the semi-empirical approach are difficult to calculate accurately, resulting in relatively limited accuracy. To achieve the optimization effect, this study proposes a method of embedding machine learning into a semi-physical empirical model (RF-PMRS). Specifically, based on the theory of the physical PM_2.5 remote sensing approach (PMRS), the complex parameter (VE_f, a columnar volume-to-extinction ratio of fine particles) is simulated by the random forest model (RF). Also, a fine mode fraction product with higher quality is applied to make up for the insufficient coverage of satellite products. Experiments in North China show that the surface PM_2.5 concentration derived by RF-PMRS has an average annual value of 57.92 μg/m³ versus the ground value of 60.23 μg/m³. Compared with the original method, RMSE decreases by 39.95 μg/m³, and the relative deviation reduces by 44.87%. Moreover, validation at two AERONET sites presents a trend closer to the true values, with an R of about 0.80. This study is also a preliminary attempt to combine model-driven and data-driven models, laying a foundation for further atmospheric research on optimization methods.

Received: 17 Sep 2022 – Discussion started: 27 Oct 2022

Caiyi Jin et al.

Status: final response (author comments only)

CC1:
'Comment on egusphere-2022-946', Adelaide Plaza., 27 Oct 2022

Very interesting! It is valuable to incorporate deep learning into the physical model.

Citation: https://doi.org/10.5194/egusphere-2022-946-CC1
- AC2: 'Reply on CC1', Qianqqiang Yuan, 27 Oct 2022
  
  Thanks for your support.
  
  Citation: https://doi.org/10.5194/egusphere-2022-946-AC2
CC2:
'Comment on egusphere-2022-946', Adelaide Plaza., 27 Oct 2022

Very interesting! It is valuable to incorporate deep learning into the physical model.

Citation: https://doi.org/10.5194/egusphere-2022-946-CC2
- AC1: 'Reply on CC2', Qianqqiang Yuan, 27 Oct 2022
  
  Thanks for your support.
  
  Citation: https://doi.org/10.5194/egusphere-2022-946-AC1
RC1:
'Comment on egusphere-2022-946', Anonymous Referee #1, 14 Nov 2022

The paper presents an optimized ML approach to estimate complex physical parameters using remote sensing data. The ML method and results are described well, and I recommend acceptance.

Citation: https://doi.org/10.5194/egusphere-2022-946-RC1
- AC3: 'Reply on RC1', Qianqqiang Yuan, 14 Nov 2022
  
  Thank the reviewer for acknowledging our work, we will further explore the combination of atmospheric mechanism and machine learning on the PM_2.5 retrieval methods.
  
  Citation: https://doi.org/10.5194/egusphere-2022-946-AC3
RC2:
'Comment on egusphere-2022-946', Anonymous Referee #2, 29 Nov 2022

General comments:

Jin et al. presented a semi-empirical physical approach for PM_2.5 retrieval from AOD and the validation results in China. The authors embedded the random forest (RF) model into the physical PM_2.5 remote sensing approach (PMRS) with a high-quality fine mode fraction product to estimate surface PM_2.5 concentration. Compared to the PMRS method, the machine learning embedded approach (PMRS-RF) showed better performance in PM_2.5 estimation, with lower biases and RMSE. The methods of the combination of machine learning and semi-empirical physical approach could be of high interest to the community, and they would be useful for PM_2.5 retrieval from satellite observations, particularly for regions with sparse ground measurements. The paper is well-written, and the ideas are presented clearly. However, the structure and the experiment designs are a bit challenging to follow. The uncertainties of the machine learning approach (e.g., out-of-sample performance) and spatial distributions of biases can be discussed more.

1. It will be better to separate data and experiment results into two sections. I suggest the authors move the data section before the method section, as some variables or datasets are mentioned in the method section (e.g., Phy-DL FMF dataset). I think a better layout will be Data as the second section, Method as the third, and Results as the fourth.

2. There are many experiments, but they are not presented in a clear way. If I understand them correctly, there are 1) a 10-fold CV and hold-out test (not sure for which year) for VE_f validation, 2) a hold-out test of 2017 for PM_2.5 validation at Beijing and Beijing-CAMS sites, and 3) a generalization test for PM_2.5 validation within North China (not sure for which year). In addition, is it correct that AERONET AOD is used for calculating PM_2.5 concentration for the experiments of BJ and BC while MODIS AOD is used for North China data? It will be better to include a table or state these experiments clearly.

3. Validation selection: In section 3.2.2, the authors selected 2017 as the validation. I wonder why this year was selected as a validation year. Any characteristics? Also, was VE_f based on RF obtained from the hold-out experiment (i.e., using data except or before 2017 at BJ and BC as training and 2017 at BJ and BC as testing) or 10-fold cross-validation? The experiment year of VE_f and surface PM_2.5 should be consistent.

4. The temporal scale (daily or hourly?), study period, and study regions are not stated clearly. Maybe the authors could include this information along with the experiments I mentioned in comment #2.

Specific comments:

1. Page 4, line 106: Please consider moving the method section after the data section.

2. Page 8, Table 1: This table should be with the data section of AERONET; the data section should be presented before the method section.

3. Page 9, line 216: How does the difference between station FMF and Phy-DL FMF influence surface PM_2.5 estimation?

4. Page 9, line 232: Please consider separate results from this section.

5. Page 10, line 247: Please include more information about the Phy-DL FMF dataset, as it is one of the important components of this paper. How did you calculate or derive FMF in this dataset? What are the differences between FMF in this dataset and at the AERONET sites?

6. Page 10, line 257: It seems like the spatial resolutions of AOD, FMF, and ERA5 meteorology are different. How do different spatial resolutions affect PM2.5 estimation? Please elaborate the uncertainties of various resolutions of the input data.

7. Page 11, line 270: Is AERONET AOD used for calculating PM_2.5 concentration for the experiments of BJ and BC, while MODIS AOD is used for North China? If so, how do the differences between two AOD products affect PM_2.5 estimation? Suppose this approach would be applied to regions where AERONET is not available (the most likely scenario); it is important to evaluate the biases caused by different AOD products, particularly the input variables of RF are based on AERONET data.

8. Page 12, Fig. 3: Please mark the AERONET sites (Beijing and Beijing-CAMS) on the map (use different colors and shapes).

9. Page 12, line 288: The experiment period is a little bit confusing. The surface PM_2.5 validation is conducted for 2017, while the VE_f validation is based on the 10-fold CV and different hold-out periods. Also, please justify the test selection for 2017.

10. Page 13, lines 308-309: Was the VE_f based on RF derived from the hold-out experiment? Ideally, the VE_f based on RF should be from test results (i.e., using data in Table 1 but excluding data at BJ and BC in 2017 as training and data at BJ and BC in 2017 as testing).

11. Page 14, Fig. 4: What is the correlation between STA and PMRS (RF-PMRS) and the RMSE or bias of the time series?

12. Page 16, line 361: In the RF model estimating VE_f, the authors include longitude and latitude as predictors, while the longitude and latitude of the sites in North China are out of training samples (Table 1). How can we trust the extrapolation of the RF model (and technically, RF is bad at extrapolation)?

13. Page 16, line 361: This section mainly discusses the general performance comparison between PMRS and RF-PMRS. It would be helpful if the authors could elaborate more about the spatial or temporal distribution of biases for the two methods (e.g., which area or period shows larger improvement and why; what are the associated factors influencing VE_f).

14. Page 17, lines 382-385: What do the high-value points mean? The high values of PM_2.5 concentration or VE_f? I guess the underestimation of VEf would lead to the underestimation of PM_2.5 in RF-RMRS.

15. Page 18, line 414: Is this experiment also based on 2017 and North China? Please specify.

16. Page 19, line 430: Is this experiment also based on 2017 and North China? Please specify.

17. Page 20, line 448: The authors should consider adding more discussions, including 1) uncertainties of the embedded RF approach (e.g., out-of-sample issue mentioned in comment #12 and the uncertainties of PM_2.5 estimation associated with different data sources), 2) spatial or temporal distribution of biases for the two methods (see comment #13).

Technical comments:

1. Page 2, line 37 & Page 13, line 314: The word “trends” is misused. Fig. 4 displays the “time series” of PM_2.5 values in 2017. In my opinion, “trends” is often used to describe a long-term increase or decrease in the data, which is not the case in Fig. 4.

2. Page 19, line 419: Please specify DOY in the main text.

Citation: https://doi.org/10.5194/egusphere-2022-946-RC2
- AC4:
  'Reply on RC2', Qianqqiang Yuan, 24 Dec 2022
  We would like to take this opportunity to gratefully thank the reviewer for his/her constructive suggestions for improving the paper. According to the comments, we will make further adjustments to our manuscript:
  1) Change the structure of the article and optimize the experimental expression, to make the expression clearer.
  2) Add more discussion to make the experiment more reliable (such as the mentioned machine learning method and experimental results).
  
  The major revisions will include:
  1) Adjust the article structure: the suggested structure is "data-method-result".
  2) Clarify the expression of the design experiment: add tables or statements.
  3) Clarify the relevant statements of the article: such as the time scale, time range, and experimental area.
  4) Consider adding more discussion, including:
  Uncertainty of embedded RF, such as problems outside the sample and uncertainty of PM_2.5 estimation related to different data sources.
  
  Spatial or temporal distribution of the deviation between the two methods.
  
  5) Change the details of the language expression and drawing, etc.
  
  We will carefully revise the paper according to these comments, then send the specific response and revised manuscript soon. Gratefully thank the reviewer for his/her encouragement and suggestions again.
  
  Citation: https://doi.org/10.5194/egusphere-2022-946-AC4
RC3:
'Comment on egusphere-2022-946', Anonymous Referee #3, 03 Jan 2023
This manuscript used the Random Forest machine learning method to improve the calculation of the parameter VEf, which is the columnar volume-to-extinction ratio of fine particles, in order to improve the PM2.5 simulation. The results present the new method outperformed the traditional method. In addition, this study combined the model-based and observation-based data to further improve the accuracy of PM2.5 simulation. However, this manuscript should have better organized the structure. The data, method and result sections should have the clear clue. Second, eq. 3 is used to calculate the ground truth of VEf, but why VEf should be estimated by the PMRS (eq. 8) and the RF method, which is very confused for the logic of this study. The RF model is trained by the spatial and temporal variables, leading to that the relationship will depend on the different locations. As a consequence, the ML model may not represent the intrinsic physical relationship. Third, the result section only selects two similar sites to estimate the performance. It could not be enough and could be better using more sites with different aerosol types. Finally, there are several obvious typos in the manuscript, and the English language is poor. I think the authors should be asked to have the manuscript proofread by a native English speaker before the article can be considered for publication in a scientific journal. Therefore, I would recommend for a major revision.

Mirror issues:

Line 62-64: The authors say the machine learning is the powerful tool. But in the next sentence, you write “the regression is affected by the distribution and density of ground stations”. It is confused that what the challenge is for the ML method. Is it correct the ML methods cannot achieve better performance for the second category methods? Do you try to compare your method with these methods?

Line 75: It is confused what the relationship between the sentence “PM2.5 concentration was estimated … ” and the previous half sentence is?

Line 97: RF is not a deep learning method.

Line 133: what does the “PVSD” stand for?

Line 148: please add the reference for the Eq 8.

Line 152: how to define the “uncertainty”?

Line 155: You mentioned that aerosol type and spatiotemporal variables could affect the regression. It could be better to discuss their importance in the result section.

Line 173-178: this paragraph should be rephrased. It could be more clear if you firstly present the problem of the original FMF dataset and then introduce the benefit of the phy-DL FMF dataset, including the details of this dataset.

Line 177: what does the “spatiotemporal continuity” stand for?

Line 192: it is confused here why don’t use eq 3-5 to directly calculate the PM2.5?

The title of step 1-4 should be more clear.

Line 212: use the correct reference for RF

Line 214: The introduce of RF is not clear. RF doesn’t learn in the random manner.

Line 221: briefly introduce the cross-validation and isolated-validation.

Line 235: add the reference for MCD19A2.

Line 287: how to align the location for different source dataset?

Line 289: does the empirical value introduce uncertainty?

Table 2: which sites the performance statistics are for? Do you try to compare the performance with the polynomial regression?

Line 309: why do you choose these only two sites?

Line 367: could you explain why the difference is not significant in terms of R, but there is a big difference for RMSE and MAE?

Line 383: you mentioned the underestimation could be caused by the high-value points. It seems that the RF model is overfitting. Do you try to reduce the overfitting?
Citation: https://doi.org/10.5194/egusphere-2022-946-RC3
- AC5: 'Reply on RC3', Qianqqiang Yuan, 08 Jan 2023
  
  Thank the reviewer for his/her constructive comments. According to these suggestions, we will make major adjustments and modifications in the article structure, experimental logic, and English grammar. We will carefully revise the paper, then send the specific response and revised manuscript soon. Gratefully thank the reviewer again.
  
  Citation: https://doi.org/10.5194/egusphere-2022-946-AC5

Caiyi Jin et al.

Viewed

Total article views: 691 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
534	131	26	691	4	6

HTML: 534
PDF: 131
XML: 26
Total: 691
BibTeX: 4
EndNote: 6

Views and downloads (calculated since 27 Oct 2022)

Month	HTML	PDF	XML	Total
Oct 2022	145	35	12	192
Nov 2022	152	47	6	205
Dec 2022	78	16	2	96
Jan 2023	65	11	4	80
Feb 2023	61	19	0	80
Mar 2023	33	3	2	38

Cumulative views and downloads (calculated since 27 Oct 2022)

Month	HTML	PDF	XML	Total
Oct 2022	145	35	12	192
Nov 2022	152	47	6	205
Dec 2022	78	16	2	96
Jan 2023	65	11	4	80
Feb 2023	61	19	0	80
Mar 2023	33	3	2	38

Viewed (geographical distribution)

Total article views: 678 (including HTML, PDF, and XML) Thereof 678 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 22 Mar 2023

Short summary

The semi-empirical physical approach derives PM_2.5 with strong physical significance. However, due to the complex optical characteristic, the physical parameters are difficult to express accurately. Thus, combining the atmospheric physical mechanism and machine learning, we propose an optimized model. It creatively embeds the random forest model into the physical PM_2.5 remote sensing approach to simulate a physical parameter. Our method shows great optimized performance in the validations.


Total:	0
HTML:	0
PDF:	0
XML:	0

An optimized semi-empirical physical approach for satellite-based PM2.5 retrieval: embedding machine learning to simulate complex physical parameters

Viewed

Viewed (geographical distribution)

An optimized semi-empirical physical approach for satellite-based PM_2.5 retrieval: embedding machine learning to simulate complex physical parameters