An optimized semi-empirical physical approach for satellite-based PM<sub>2.5</sub> retrieval: embedding machine learning to simulate complex physical parameters

Jin, Caiyi; Yuan, Qiangqiang; Li, Tongwen; Wang, Yuan; Zhang, Liangpei

doi:https://doi.org/10.5194/egusphere-2022-946

Preprints

https://doi.org/10.5194/egusphere-2022-946

Preprints

27 Oct 2022

| 27 Oct 2022

An optimized semi-empirical physical approach for satellite-based PM_2.5 retrieval: embedding machine learning to simulate complex physical parameters

Caiyi Jin, Qiangqiang Yuan, Tongwen Li, Yuan Wang, and Liangpei Zhang

Abstract. Satellite remote sensing of PM_2.5 mass concentration has become one of the most popular atmospheric research aspects, resulting in the development of different models. Among them, the semi-empirical physical approach constructs the transformation relationship between the aerosol optical depth (AOD) and PM_2.5 based on the optical properties of particles, which has strong physical significance. Also, it performs the PM_2.5 retrieval independently of the ground stations. However, due to the complex physical relationship, the physical parameters in the semi-empirical approach are difficult to calculate accurately, resulting in relatively limited accuracy. To achieve the optimization effect, this study proposes a method of embedding machine learning into a semi-physical empirical model (RF-PMRS). Specifically, based on the theory of the physical PM_2.5 remote sensing approach (PMRS), the complex parameter (VE_f, a columnar volume-to-extinction ratio of fine particles) is simulated by the random forest model (RF). Also, a fine mode fraction product with higher quality is applied to make up for the insufficient coverage of satellite products. Experiments in North China show that the surface PM_2.5 concentration derived by RF-PMRS has an average annual value of 57.92 μg/m³ versus the ground value of 60.23 μg/m³. Compared with the original method, RMSE decreases by 39.95 μg/m³, and the relative deviation reduces by 44.87%. Moreover, validation at two AERONET sites presents a trend closer to the true values, with an R of about 0.80. This study is also a preliminary attempt to combine model-driven and data-driven models, laying a foundation for further atmospheric research on optimization methods.

Received: 17 Sep 2022 – Discussion started: 27 Oct 2022

Download & links

Preprint (PDF, 4186 KB)

Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
Preprint (4186 KB)

Download & links

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Journal article(s) based on this preprint

24 Jul 2023

An optimized semi-empirical physical approach for satellite-based PM_2.5 retrieval: embedding machine learning to simulate complex physical parameters

Caiyi Jin, Qiangqiang Yuan, Tongwen Li, Yuan Wang, and Liangpei Zhang

Geosci. Model Dev., 16, 4137–4154, https://doi.org/10.5194/gmd-16-4137-2023,https://doi.org/10.5194/gmd-16-4137-2023, 2023

Short summary

Caiyi Jin et al.

Interactive discussion

Status: closed

CC1:
'Comment on egusphere-2022-946', Adelaide Plaza., 27 Oct 2022

Very interesting! It is valuable to incorporate deep learning into the physical model.

Citation: https://doi.org/10.5194/egusphere-2022-946-CC1
- AC2: 'Reply on CC1', Qianqqiang Yuan, 27 Oct 2022
  
  Thanks for your support.
  
  Citation: https://doi.org/10.5194/egusphere-2022-946-AC2
CC2:
'Comment on egusphere-2022-946', Adelaide Plaza., 27 Oct 2022

Very interesting! It is valuable to incorporate deep learning into the physical model.

Citation: https://doi.org/10.5194/egusphere-2022-946-CC2
- AC1: 'Reply on CC2', Qianqqiang Yuan, 27 Oct 2022
  
  Thanks for your support.
  
  Citation: https://doi.org/10.5194/egusphere-2022-946-AC1
RC1:
'Comment on egusphere-2022-946', Anonymous Referee #1, 14 Nov 2022

The paper presents an optimized ML approach to estimate complex physical parameters using remote sensing data. The ML method and results are described well, and I recommend acceptance.

Citation: https://doi.org/10.5194/egusphere-2022-946-RC1
- AC3: 'Reply on RC1', Qianqqiang Yuan, 14 Nov 2022
  
  Thank the reviewer for acknowledging our work, we will further explore the combination of atmospheric mechanism and machine learning on the PM_2.5 retrieval methods.
  
  Citation: https://doi.org/10.5194/egusphere-2022-946-AC3
RC2:
'Comment on egusphere-2022-946', Anonymous Referee #2, 29 Nov 2022

General comments:

Jin et al. presented a semi-empirical physical approach for PM_2.5 retrieval from AOD and the validation results in China. The authors embedded the random forest (RF) model into the physical PM_2.5 remote sensing approach (PMRS) with a high-quality fine mode fraction product to estimate surface PM_2.5 concentration. Compared to the PMRS method, the machine learning embedded approach (PMRS-RF) showed better performance in PM_2.5 estimation, with lower biases and RMSE. The methods of the combination of machine learning and semi-empirical physical approach could be of high interest to the community, and they would be useful for PM_2.5 retrieval from satellite observations, particularly for regions with sparse ground measurements. The paper is well-written, and the ideas are presented clearly. However, the structure and the experiment designs are a bit challenging to follow. The uncertainties of the machine learning approach (e.g., out-of-sample performance) and spatial distributions of biases can be discussed more.

1. It will be better to separate data and experiment results into two sections. I suggest the authors move the data section before the method section, as some variables or datasets are mentioned in the method section (e.g., Phy-DL FMF dataset). I think a better layout will be Data as the second section, Method as the third, and Results as the fourth.

2. There are many experiments, but they are not presented in a clear way. If I understand them correctly, there are 1) a 10-fold CV and hold-out test (not sure for which year) for VE_f validation, 2) a hold-out test of 2017 for PM_2.5 validation at Beijing and Beijing-CAMS sites, and 3) a generalization test for PM_2.5 validation within North China (not sure for which year). In addition, is it correct that AERONET AOD is used for calculating PM_2.5 concentration for the experiments of BJ and BC while MODIS AOD is used for North China data? It will be better to include a table or state these experiments clearly.

3. Validation selection: In section 3.2.2, the authors selected 2017 as the validation. I wonder why this year was selected as a validation year. Any characteristics? Also, was VE_f based on RF obtained from the hold-out experiment (i.e., using data except or before 2017 at BJ and BC as training and 2017 at BJ and BC as testing) or 10-fold cross-validation? The experiment year of VE_f and surface PM_2.5 should be consistent.

4. The temporal scale (daily or hourly?), study period, and study regions are not stated clearly. Maybe the authors could include this information along with the experiments I mentioned in comment #2.

Specific comments:

1. Page 4, line 106: Please consider moving the method section after the data section.

2. Page 8, Table 1: This table should be with the data section of AERONET; the data section should be presented before the method section.

3. Page 9, line 216: How does the difference between station FMF and Phy-DL FMF influence surface PM_2.5 estimation?

4. Page 9, line 232: Please consider separate results from this section.

5. Page 10, line 247: Please include more information about the Phy-DL FMF dataset, as it is one of the important components of this paper. How did you calculate or derive FMF in this dataset? What are the differences between FMF in this dataset and at the AERONET sites?

6. Page 10, line 257: It seems like the spatial resolutions of AOD, FMF, and ERA5 meteorology are different. How do different spatial resolutions affect PM2.5 estimation? Please elaborate the uncertainties of various resolutions of the input data.

7. Page 11, line 270: Is AERONET AOD used for calculating PM_2.5 concentration for the experiments of BJ and BC, while MODIS AOD is used for North China? If so, how do the differences between two AOD products affect PM_2.5 estimation? Suppose this approach would be applied to regions where AERONET is not available (the most likely scenario); it is important to evaluate the biases caused by different AOD products, particularly the input variables of RF are based on AERONET data.

8. Page 12, Fig. 3: Please mark the AERONET sites (Beijing and Beijing-CAMS) on the map (use different colors and shapes).

9. Page 12, line 288: The experiment period is a little bit confusing. The surface PM_2.5 validation is conducted for 2017, while the VE_f validation is based on the 10-fold CV and different hold-out periods. Also, please justify the test selection for 2017.

10. Page 13, lines 308-309: Was the VE_f based on RF derived from the hold-out experiment? Ideally, the VE_f based on RF should be from test results (i.e., using data in Table 1 but excluding data at BJ and BC in 2017 as training and data at BJ and BC in 2017 as testing).

11. Page 14, Fig. 4: What is the correlation between STA and PMRS (RF-PMRS) and the RMSE or bias of the time series?

12. Page 16, line 361: In the RF model estimating VE_f, the authors include longitude and latitude as predictors, while the longitude and latitude of the sites in North China are out of training samples (Table 1). How can we trust the extrapolation of the RF model (and technically, RF is bad at extrapolation)?

13. Page 16, line 361: This section mainly discusses the general performance comparison between PMRS and RF-PMRS. It would be helpful if the authors could elaborate more about the spatial or temporal distribution of biases for the two methods (e.g., which area or period shows larger improvement and why; what are the associated factors influencing VE_f).

14. Page 17, lines 382-385: What do the high-value points mean? The high values of PM_2.5 concentration or VE_f? I guess the underestimation of VEf would lead to the underestimation of PM_2.5 in RF-RMRS.

15. Page 18, line 414: Is this experiment also based on 2017 and North China? Please specify.

16. Page 19, line 430: Is this experiment also based on 2017 and North China? Please specify.

17. Page 20, line 448: The authors should consider adding more discussions, including 1) uncertainties of the embedded RF approach (e.g., out-of-sample issue mentioned in comment #12 and the uncertainties of PM_2.5 estimation associated with different data sources), 2) spatial or temporal distribution of biases for the two methods (see comment #13).

Technical comments:

1. Page 2, line 37 & Page 13, line 314: The word “trends” is misused. Fig. 4 displays the “time series” of PM_2.5 values in 2017. In my opinion, “trends” is often used to describe a long-term increase or decrease in the data, which is not the case in Fig. 4.

2. Page 19, line 419: Please specify DOY in the main text.

Citation: https://doi.org/10.5194/egusphere-2022-946-RC2
- AC4:
  'Reply on RC2', Qianqqiang Yuan, 24 Dec 2022
  We would like to take this opportunity to gratefully thank the reviewer for his/her constructive suggestions for improving the paper. According to the comments, we will make further adjustments to our manuscript:
  1) Change the structure of the article and optimize the experimental expression, to make the expression clearer.
  2) Add more discussion to make the experiment more reliable (such as the mentioned machine learning method and experimental results).
  
  The major revisions will include:
  1) Adjust the article structure: the suggested structure is "data-method-result".
  2) Clarify the expression of the design experiment: add tables or statements.
  3) Clarify the relevant statements of the article: such as the time scale, time range, and experimental area.
  4) Consider adding more discussion, including:
  Uncertainty of embedded RF, such as problems outside the sample and uncertainty of PM_2.5 estimation related to different data sources.
  
  Spatial or temporal distribution of the deviation between the two methods.
  
  5) Change the details of the language expression and drawing, etc.
  
  We will carefully revise the paper according to these comments, then send the specific response and revised manuscript soon. Gratefully thank the reviewer for his/her encouragement and suggestions again.
  
  Citation: https://doi.org/10.5194/egusphere-2022-946-AC4
RC3:
'Comment on egusphere-2022-946', Anonymous Referee #3, 03 Jan 2023
This manuscript used the Random Forest machine learning method to improve the calculation of the parameter VEf, which is the columnar volume-to-extinction ratio of fine particles, in order to improve the PM2.5 simulation. The results present the new method outperformed the traditional method. In addition, this study combined the model-based and observation-based data to further improve the accuracy of PM2.5 simulation. However, this manuscript should have better organized the structure. The data, method and result sections should have the clear clue. Second, eq. 3 is used to calculate the ground truth of VEf, but why VEf should be estimated by the PMRS (eq. 8) and the RF method, which is very confused for the logic of this study. The RF model is trained by the spatial and temporal variables, leading to that the relationship will depend on the different locations. As a consequence, the ML model may not represent the intrinsic physical relationship. Third, the result section only selects two similar sites to estimate the performance. It could not be enough and could be better using more sites with different aerosol types. Finally, there are several obvious typos in the manuscript, and the English language is poor. I think the authors should be asked to have the manuscript proofread by a native English speaker before the article can be considered for publication in a scientific journal. Therefore, I would recommend for a major revision.

Mirror issues:

Line 62-64: The authors say the machine learning is the powerful tool. But in the next sentence, you write “the regression is affected by the distribution and density of ground stations”. It is confused that what the challenge is for the ML method. Is it correct the ML methods cannot achieve better performance for the second category methods? Do you try to compare your method with these methods?

Line 75: It is confused what the relationship between the sentence “PM2.5 concentration was estimated … ” and the previous half sentence is?

Line 97: RF is not a deep learning method.

Line 133: what does the “PVSD” stand for?

Line 148: please add the reference for the Eq 8.

Line 152: how to define the “uncertainty”?

Line 155: You mentioned that aerosol type and spatiotemporal variables could affect the regression. It could be better to discuss their importance in the result section.

Line 173-178: this paragraph should be rephrased. It could be more clear if you firstly present the problem of the original FMF dataset and then introduce the benefit of the phy-DL FMF dataset, including the details of this dataset.

Line 177: what does the “spatiotemporal continuity” stand for?

Line 192: it is confused here why don’t use eq 3-5 to directly calculate the PM2.5?

The title of step 1-4 should be more clear.

Line 212: use the correct reference for RF

Line 214: The introduce of RF is not clear. RF doesn’t learn in the random manner.

Line 221: briefly introduce the cross-validation and isolated-validation.

Line 235: add the reference for MCD19A2.

Line 287: how to align the location for different source dataset?

Line 289: does the empirical value introduce uncertainty?

Table 2: which sites the performance statistics are for? Do you try to compare the performance with the polynomial regression?

Line 309: why do you choose these only two sites?

Line 367: could you explain why the difference is not significant in terms of R, but there is a big difference for RMSE and MAE?

Line 383: you mentioned the underestimation could be caused by the high-value points. It seems that the RF model is overfitting. Do you try to reduce the overfitting?
Citation: https://doi.org/10.5194/egusphere-2022-946-RC3
- AC5: 'Reply on RC3', Qianqqiang Yuan, 08 Jan 2023
  
  Thank the reviewer for his/her constructive comments. According to these suggestions, we will make major adjustments and modifications in the article structure, experimental logic, and English grammar. We will carefully revise the paper, then send the specific response and revised manuscript soon. Gratefully thank the reviewer again.
  
  Citation: https://doi.org/10.5194/egusphere-2022-946-AC5

Interactive discussion

Status: closed

CC1:
'Comment on egusphere-2022-946', Adelaide Plaza., 27 Oct 2022

Very interesting! It is valuable to incorporate deep learning into the physical model.

Citation: https://doi.org/10.5194/egusphere-2022-946-CC1
- AC2: 'Reply on CC1', Qianqqiang Yuan, 27 Oct 2022
  
  Thanks for your support.
  
  Citation: https://doi.org/10.5194/egusphere-2022-946-AC2
CC2:
'Comment on egusphere-2022-946', Adelaide Plaza., 27 Oct 2022

Very interesting! It is valuable to incorporate deep learning into the physical model.

Citation: https://doi.org/10.5194/egusphere-2022-946-CC2
- AC1: 'Reply on CC2', Qianqqiang Yuan, 27 Oct 2022
  
  Thanks for your support.
  
  Citation: https://doi.org/10.5194/egusphere-2022-946-AC1
RC1:
'Comment on egusphere-2022-946', Anonymous Referee #1, 14 Nov 2022

The paper presents an optimized ML approach to estimate complex physical parameters using remote sensing data. The ML method and results are described well, and I recommend acceptance.

Citation: https://doi.org/10.5194/egusphere-2022-946-RC1
- AC3: 'Reply on RC1', Qianqqiang Yuan, 14 Nov 2022
  
  Thank the reviewer for acknowledging our work, we will further explore the combination of atmospheric mechanism and machine learning on the PM_2.5 retrieval methods.
  
  Citation: https://doi.org/10.5194/egusphere-2022-946-AC3
RC2:
'Comment on egusphere-2022-946', Anonymous Referee #2, 29 Nov 2022

General comments:

Jin et al. presented a semi-empirical physical approach for PM_2.5 retrieval from AOD and the validation results in China. The authors embedded the random forest (RF) model into the physical PM_2.5 remote sensing approach (PMRS) with a high-quality fine mode fraction product to estimate surface PM_2.5 concentration. Compared to the PMRS method, the machine learning embedded approach (PMRS-RF) showed better performance in PM_2.5 estimation, with lower biases and RMSE. The methods of the combination of machine learning and semi-empirical physical approach could be of high interest to the community, and they would be useful for PM_2.5 retrieval from satellite observations, particularly for regions with sparse ground measurements. The paper is well-written, and the ideas are presented clearly. However, the structure and the experiment designs are a bit challenging to follow. The uncertainties of the machine learning approach (e.g., out-of-sample performance) and spatial distributions of biases can be discussed more.

1. It will be better to separate data and experiment results into two sections. I suggest the authors move the data section before the method section, as some variables or datasets are mentioned in the method section (e.g., Phy-DL FMF dataset). I think a better layout will be Data as the second section, Method as the third, and Results as the fourth.

2. There are many experiments, but they are not presented in a clear way. If I understand them correctly, there are 1) a 10-fold CV and hold-out test (not sure for which year) for VE_f validation, 2) a hold-out test of 2017 for PM_2.5 validation at Beijing and Beijing-CAMS sites, and 3) a generalization test for PM_2.5 validation within North China (not sure for which year). In addition, is it correct that AERONET AOD is used for calculating PM_2.5 concentration for the experiments of BJ and BC while MODIS AOD is used for North China data? It will be better to include a table or state these experiments clearly.

3. Validation selection: In section 3.2.2, the authors selected 2017 as the validation. I wonder why this year was selected as a validation year. Any characteristics? Also, was VE_f based on RF obtained from the hold-out experiment (i.e., using data except or before 2017 at BJ and BC as training and 2017 at BJ and BC as testing) or 10-fold cross-validation? The experiment year of VE_f and surface PM_2.5 should be consistent.

4. The temporal scale (daily or hourly?), study period, and study regions are not stated clearly. Maybe the authors could include this information along with the experiments I mentioned in comment #2.

Specific comments:

1. Page 4, line 106: Please consider moving the method section after the data section.

2. Page 8, Table 1: This table should be with the data section of AERONET; the data section should be presented before the method section.

3. Page 9, line 216: How does the difference between station FMF and Phy-DL FMF influence surface PM_2.5 estimation?

4. Page 9, line 232: Please consider separate results from this section.

5. Page 10, line 247: Please include more information about the Phy-DL FMF dataset, as it is one of the important components of this paper. How did you calculate or derive FMF in this dataset? What are the differences between FMF in this dataset and at the AERONET sites?

6. Page 10, line 257: It seems like the spatial resolutions of AOD, FMF, and ERA5 meteorology are different. How do different spatial resolutions affect PM2.5 estimation? Please elaborate the uncertainties of various resolutions of the input data.

7. Page 11, line 270: Is AERONET AOD used for calculating PM_2.5 concentration for the experiments of BJ and BC, while MODIS AOD is used for North China? If so, how do the differences between two AOD products affect PM_2.5 estimation? Suppose this approach would be applied to regions where AERONET is not available (the most likely scenario); it is important to evaluate the biases caused by different AOD products, particularly the input variables of RF are based on AERONET data.

8. Page 12, Fig. 3: Please mark the AERONET sites (Beijing and Beijing-CAMS) on the map (use different colors and shapes).

9. Page 12, line 288: The experiment period is a little bit confusing. The surface PM_2.5 validation is conducted for 2017, while the VE_f validation is based on the 10-fold CV and different hold-out periods. Also, please justify the test selection for 2017.

10. Page 13, lines 308-309: Was the VE_f based on RF derived from the hold-out experiment? Ideally, the VE_f based on RF should be from test results (i.e., using data in Table 1 but excluding data at BJ and BC in 2017 as training and data at BJ and BC in 2017 as testing).

11. Page 14, Fig. 4: What is the correlation between STA and PMRS (RF-PMRS) and the RMSE or bias of the time series?

12. Page 16, line 361: In the RF model estimating VE_f, the authors include longitude and latitude as predictors, while the longitude and latitude of the sites in North China are out of training samples (Table 1). How can we trust the extrapolation of the RF model (and technically, RF is bad at extrapolation)?

13. Page 16, line 361: This section mainly discusses the general performance comparison between PMRS and RF-PMRS. It would be helpful if the authors could elaborate more about the spatial or temporal distribution of biases for the two methods (e.g., which area or period shows larger improvement and why; what are the associated factors influencing VE_f).

14. Page 17, lines 382-385: What do the high-value points mean? The high values of PM_2.5 concentration or VE_f? I guess the underestimation of VEf would lead to the underestimation of PM_2.5 in RF-RMRS.

15. Page 18, line 414: Is this experiment also based on 2017 and North China? Please specify.

16. Page 19, line 430: Is this experiment also based on 2017 and North China? Please specify.

17. Page 20, line 448: The authors should consider adding more discussions, including 1) uncertainties of the embedded RF approach (e.g., out-of-sample issue mentioned in comment #12 and the uncertainties of PM_2.5 estimation associated with different data sources), 2) spatial or temporal distribution of biases for the two methods (see comment #13).

Technical comments:

1. Page 2, line 37 & Page 13, line 314: The word “trends” is misused. Fig. 4 displays the “time series” of PM_2.5 values in 2017. In my opinion, “trends” is often used to describe a long-term increase or decrease in the data, which is not the case in Fig. 4.

2. Page 19, line 419: Please specify DOY in the main text.

Citation: https://doi.org/10.5194/egusphere-2022-946-RC2
- AC4:
  'Reply on RC2', Qianqqiang Yuan, 24 Dec 2022
  We would like to take this opportunity to gratefully thank the reviewer for his/her constructive suggestions for improving the paper. According to the comments, we will make further adjustments to our manuscript:
  1) Change the structure of the article and optimize the experimental expression, to make the expression clearer.
  2) Add more discussion to make the experiment more reliable (such as the mentioned machine learning method and experimental results).
  
  The major revisions will include:
  1) Adjust the article structure: the suggested structure is "data-method-result".
  2) Clarify the expression of the design experiment: add tables or statements.
  3) Clarify the relevant statements of the article: such as the time scale, time range, and experimental area.
  4) Consider adding more discussion, including:
  Uncertainty of embedded RF, such as problems outside the sample and uncertainty of PM_2.5 estimation related to different data sources.
  
  Spatial or temporal distribution of the deviation between the two methods.
  
  5) Change the details of the language expression and drawing, etc.
  
  We will carefully revise the paper according to these comments, then send the specific response and revised manuscript soon. Gratefully thank the reviewer for his/her encouragement and suggestions again.
  
  Citation: https://doi.org/10.5194/egusphere-2022-946-AC4
RC3:
'Comment on egusphere-2022-946', Anonymous Referee #3, 03 Jan 2023
This manuscript used the Random Forest machine learning method to improve the calculation of the parameter VEf, which is the columnar volume-to-extinction ratio of fine particles, in order to improve the PM2.5 simulation. The results present the new method outperformed the traditional method. In addition, this study combined the model-based and observation-based data to further improve the accuracy of PM2.5 simulation. However, this manuscript should have better organized the structure. The data, method and result sections should have the clear clue. Second, eq. 3 is used to calculate the ground truth of VEf, but why VEf should be estimated by the PMRS (eq. 8) and the RF method, which is very confused for the logic of this study. The RF model is trained by the spatial and temporal variables, leading to that the relationship will depend on the different locations. As a consequence, the ML model may not represent the intrinsic physical relationship. Third, the result section only selects two similar sites to estimate the performance. It could not be enough and could be better using more sites with different aerosol types. Finally, there are several obvious typos in the manuscript, and the English language is poor. I think the authors should be asked to have the manuscript proofread by a native English speaker before the article can be considered for publication in a scientific journal. Therefore, I would recommend for a major revision.

Mirror issues:

Line 62-64: The authors say the machine learning is the powerful tool. But in the next sentence, you write “the regression is affected by the distribution and density of ground stations”. It is confused that what the challenge is for the ML method. Is it correct the ML methods cannot achieve better performance for the second category methods? Do you try to compare your method with these methods?

Line 75: It is confused what the relationship between the sentence “PM2.5 concentration was estimated … ” and the previous half sentence is?

Line 97: RF is not a deep learning method.

Line 133: what does the “PVSD” stand for?

Line 148: please add the reference for the Eq 8.

Line 152: how to define the “uncertainty”?

Line 155: You mentioned that aerosol type and spatiotemporal variables could affect the regression. It could be better to discuss their importance in the result section.

Line 173-178: this paragraph should be rephrased. It could be more clear if you firstly present the problem of the original FMF dataset and then introduce the benefit of the phy-DL FMF dataset, including the details of this dataset.

Line 177: what does the “spatiotemporal continuity” stand for?

Line 192: it is confused here why don’t use eq 3-5 to directly calculate the PM2.5?

The title of step 1-4 should be more clear.

Line 212: use the correct reference for RF

Line 214: The introduce of RF is not clear. RF doesn’t learn in the random manner.

Line 221: briefly introduce the cross-validation and isolated-validation.

Line 235: add the reference for MCD19A2.

Line 287: how to align the location for different source dataset?

Line 289: does the empirical value introduce uncertainty?

Table 2: which sites the performance statistics are for? Do you try to compare the performance with the polynomial regression?

Line 309: why do you choose these only two sites?

Line 367: could you explain why the difference is not significant in terms of R, but there is a big difference for RMSE and MAE?

Line 383: you mentioned the underestimation could be caused by the high-value points. It seems that the RF model is overfitting. Do you try to reduce the overfitting?
Citation: https://doi.org/10.5194/egusphere-2022-946-RC3
- AC5: 'Reply on RC3', Qianqqiang Yuan, 08 Jan 2023
  
  Thank the reviewer for his/her constructive comments. According to these suggestions, we will make major adjustments and modifications in the article structure, experimental logic, and English grammar. We will carefully revise the paper, then send the specific response and revised manuscript soon. Gratefully thank the reviewer again.
  
  Citation: https://doi.org/10.5194/egusphere-2022-946-AC5

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision | EF: Editorial file upload

AR by Qianqqiang Yuan on behalf of the Authors (03 Feb 2023) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (17 Feb 2023) by Po-Lun Ma

RR by Anonymous Referee #2 (08 Mar 2023)

Suggestions for revision or reasons for rejection

The authors have addressed most of the reviewers’ comments and revised the manuscript accordingly. Their results demonstrated better performance of the machine learning embedded approach (PMRS-RF) than the PMRS approach. The additional analyses of RF feature importance helped readers understand the factors controlling VEf, and the spatial distribution of RMSE demonstrated how PMRS-RF improves PM2.5 estimations. However, I still have a few concerns, as described below:

1. Page 2, line 62-65: “Machine learning….. between multiple variables (Irrgang et al., 2021). But the regression is ….. ground stations (Gupta and Christopher, 2009; Li et al., 2017).” There is no apparent connection between the two sentences. The first part of the sentence is about machine learning, but the second sentence jumps to the regression. Please consider rewriting the sentence.

2. Page 7, Fig. 1: Please mark the BJ and BC sites with triangles or stars.

3. Page 10, line 252: As the authors mentioned in comment #2, the AERONET AOD is used in step 1, while MODIS AOD is used in step 4. How does the AOD from two sources affect the PM2.5 estimation, considering the uncertainties of the two sources?

4. Page 11, line 274: In experiment 3, the authors applied the RF to estimate VEf and PM2.5 concentration over North China. The authors used sites worldwide (Table 1) to train the RF for estimating VEf, and the relationships learned by RF are based on the training data. How can RF represent the relationships within North China based on only two sites in this region in the training data? The authors should at least include this issue and the associated uncertainties in the discussion. I think this comment was also raised by Reviewer #3, but it was not fully addressed in the response.

5. Page 12, line 283: Is station FMF calculated from equation 10?

6. Page 13, line 323: If I understand correctly, experiment 1 in Table 2 is just for model evaluation (internal and external accuracy). In experiments 2 and 3, did the authors use data except for 2017 to retrain the model and get the estimation of VEf of 2017 for PM2.5 calculation? Or the authors used the VEf of 2017 from results from the 10-fold CV? If it's the second one, I doubt the consistency of the experiment, as mentioned before. The authors should clearly describe how they obtained the VEf for experiment 2 (3) at the beginning of Section 4.2 (4.3).

7. Page 13, line 324: Please specify when you applied Phy-DL FMF in the process. It wasn't very clear to me. The authors mentioned that station FMF was used in VEf true value calculation and RF training. It was not clear when Phy-DL was applied.

8. Page 17, line 380: In experiment 3, I guess the author also used Phy DL FMF to replace S-FMF in the RF to derive the VEf since there is no S-FMF data for each site in North China. In this case, how does the difference between station FMF and Phy-DL affect the VEf and PM2.5 estimation?

9. Page 20, line 453 (Table 5): Maybe the comparison would be valid when the PMRS with MODIS and Phy-DL FMF were sampled on the same days. It seems like the statistics were based on two sets of days.

10. Page 20, Section 5: After reading the revised manuscript, I now understand the flow and structure of this study better. It seems to me the (same) parameters from different sources are often used to estimate surface PM2.5 (e.g., S-FMF vs. Phy-DL FMF and AERONET AOD vs. MODIS AOD). I am confused about when and where they are used. Also, the differences in data used in training and generalization (e.g., S-FMF used in training vs. Phy-DL FMF used in generalization (North China) for VEf prediction) and the associated biases would propagate to PM2.5 estimation. How do you quantify the uncertainties? It is critical to include this issue in the discussion.

Hide

ED: Reconsider after major revisions (22 Mar 2023) by Po-Lun Ma

Dear Caiyi Yin,

The reviewer acknowledged that most comments have been addressed, but there are still some issues that need your attention:

1. Page 2, line 62-65: “Machine learning….. between multiple variables (Irrgang et al., 2021). But the regression is ….. ground stations (Gupta and Christopher, 2009; Li et al., 2017).” There is no apparent connection between the two sentences. The first part of the sentence is about machine learning, but the second sentence jumps to the regression. Please consider rewriting the sentence.

2. Page 7, Fig. 1: Please mark the BJ and BC sites with triangles or stars.

3. Page 10, line 252: As the authors mentioned in comment #2, the AERONET AOD is used in step 1, while MODIS AOD is used in step 4. How does the AOD from two sources affect the PM2.5 estimation, considering the uncertainties of the two sources?

4. Page 11, line 274: In experiment 3, the authors applied the RF to estimate VEf and PM2.5 concentration over North China. The authors used sites worldwide (Table 1) to train the RF for estimating VEf, and the relationships learned by RF are based on the training data. How can RF represent the relationships within North China based on only two sites in this region in the training data? The authors should at least include this issue and the associated uncertainties in the discussion. I think this comment was also raised by Reviewer #3, but it was not fully addressed in the response.

5. Page 12, line 283: Is station FMF calculated from equation 10?

6. Page 13, line 323: If I understand correctly, experiment 1 in Table 2 is just for model evaluation (internal and external accuracy). In experiments 2 and 3, did the authors use data except for 2017 to retrain the model and get the estimation of VEf of 2017 for PM2.5 calculation? Or the authors used the VEf of 2017 from results from the 10-fold CV? If it's the second one, I doubt the consistency of the experiment, as mentioned before. The authors should clearly describe how they obtained the VEf for experiment 2 (3) at the beginning of Section 4.2 (4.3).

7. Page 13, line 324: Please specify when you applied Phy-DL FMF in the process. It wasn't very clear to me. The authors mentioned that station FMF was used in VEf true value calculation and RF training. It was not clear when Phy-DL was applied.

8. Page 17, line 380: In experiment 3, I guess the author also used Phy DL FMF to replace S-FMF in the RF to derive the VEf since there is no S-FMF data for each site in North China. In this case, how does the difference between station FMF and Phy-DL affect the VEf and PM2.5 estimation?

9. Page 20, line 453 (Table 5): Maybe the comparison would be valid when the PMRS with MODIS and Phy-DL FMF were sampled on the same days. It seems like the statistics were based on two sets of days.

10. Page 20, Section 5: After reading the revised manuscript, I now understand the flow and structure of this study better. It seems to me the (same) parameters from different sources are often used to estimate surface PM2.5 (e.g., S-FMF vs. Phy-DL FMF and AERONET AOD vs. MODIS AOD). I am confused about when and where they are used. Also, the differences in data used in training and generalization (e.g., S-FMF used in training vs. Phy-DL FMF used in generalization (North China) for VEf prediction) and the associated biases would propagate to PM2.5 estimation. How do you quantify the uncertainties? It is critical to include this issue in the discussion.

Please address these comments accordingly.

Regards,

Po-Lun

Hide

AR by Qianqqiang Yuan on behalf of the Authors (19 Apr 2023) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (29 Apr 2023) by Po-Lun Ma

RR by Anonymous Referee #2 (28 May 2023)

Suggestions for revision or reasons for rejection

I appreciate the authors’ efforts in addressing the comments and clarifying the data used in the experiments. The revised manuscript is clearer, and the discussions of uncertainties of methodologies and data sources are helpful for future use of the RF-PMRS. However, there are still some points that need clarification before publication:

1. Lines 73-75: The sentence is confusing and may contain a grammatical error. It may be helpful to include a comma between ‘S’ and ‘and’ in order to improve clarity. The suggested sentence would be: “Based on 355nm-band radar observations, Raut and Chazette (2009) introduced a specific extinction cross-section to simplify the expression of S, and the PM2.5 concentration was estimated.”

2. Line 294: It could be revised to “Then, we calculate….” The use of imperative sentences in the manuscript is not common or formal in academic writing.

3. Lines 335-336: Does it mean you feed Phy-DL FMF to the trained model, which is based on S-FMF? If so, I would suggest conducting an additional experiment to test the performance of VEf using S-FMF and Phy-DL FMF under experiment 1 (e.g., Isolated-validation with inputs of Phy-DL FMF) and including the results in Section 5.4.3.

4. Line 477, Table 5: It will be helpful to include a comparison based on the same valid DOY (e.g., the intersection of valid DOY of MODIS FMF and Phy-DL FMF) to ensure a fair comparison.

5. Lines 532-534: This sentence “Analyzing the model construction…..” is difficult to read and understand. I recommend revising it for better clarity and flow. There are similar issues with other sentences in the manuscript that the authors may want to address before publication.

6. Lines 573-574: It would be helpful to include a comparison between Phy-DL FMF and S-FMF to support the statement made here. Alternatively, references could be included to provide additional support.

Hide

RR by Anonymous Referee #3 (01 Jun 2023)

ED: Publish subject to minor revisions (review by editor) (08 Jun 2023) by Po-Lun Ma

Dear Qianqqiang Yuan,

I now have received the reports from two reviewers. Overall they consider this manuscript greatly improved but point out several places that can use some clarification. I invite you to consider their suggestions and make clarifications as needed, and submit the revised manuscript soon. I list their comments below:

From Reviewer #2:

1. Lines 73-75: The sentence is confusing and may contain a grammatical error. It may be helpful to include a comma between ‘S’ and ‘and’ in order to improve clarity. The suggested sentence would be: “Based on 355nm-band radar observations, Raut and Chazette (2009) introduced a specific extinction cross-section to simplify the expression of S, and the PM2.5 concentration was estimated.”

2. Line 294: It could be revised to “Then, we calculate….” The use of imperative sentences in the manuscript is not common or formal in academic writing.

3. Lines 335-336: Does it mean you feed Phy-DL FMF to the trained model, which is based on S-FMF? If so, I would suggest conducting an additional experiment to test the performance of VEf using S-FMF and Phy-DL FMF under experiment 1 (e.g., Isolated-validation with inputs of Phy-DL FMF) and including the results in Section 5.4.3.

4. Line 477, Table 5: It will be helpful to include a comparison based on the same valid DOY (e.g., the intersection of valid DOY of MODIS FMF and Phy-DL FMF) to ensure a fair comparison.

5. Lines 532-534: This sentence “Analyzing the model construction…..” is difficult to read and understand. I recommend revising it for better clarity and flow. There are similar issues with other sentences in the manuscript that the authors may want to address before publication.

6. Lines 573-574: It would be helpful to include a comparison between Phy-DL FMF and S-FMF to support the statement made here. Alternatively, references could be included to provide additional support.

From Reviewer #3:

Comment #1: The authors may not correctly understand my comment. In the original introduction, they introduce three kinds of method to calculate PM2.5. The second one is the univariate/multivariate regression. The third one is the semi-empirical ML approach. The second one could be the baseline. Although the semi-empirical method is the physics informed, it could be necessary to compare it with the baseline to prove that the accuracy is better than the end-to-end ML method.

Comment #7: The authors didn’t reply how the aerosol type affect the prediction. They mentioned several times about the aerosol type. It could be the important factor for final PM2.5.

Comment #12: The correct reference for RF should be “Ho, Tin Kam. "Random decision forests." In Proceedings of 3rd international conference on document analysis and recognition, vol. 1, pp. 278-282. IEEE, 1995.”

Comment #16: I think it could be helpful if the authors add the discussion in the manuscript.

Regards,

Po-Lun

Hide

AR by Qianqqiang Yuan on behalf of the Authors (16 Jun 2023) Author's response Author's tracked changes Manuscript

ED: Publish as is (22 Jun 2023) by Po-Lun Ma

AR by Qianqqiang Yuan on behalf of the Authors (26 Jun 2023) Author's response Manuscript

Journal article(s) based on this preprint

24 Jul 2023

An optimized semi-empirical physical approach for satellite-based PM_2.5 retrieval: embedding machine learning to simulate complex physical parameters

Caiyi Jin, Qiangqiang Yuan, Tongwen Li, Yuan Wang, and Liangpei Zhang

Geosci. Model Dev., 16, 4137–4154, https://doi.org/10.5194/gmd-16-4137-2023,https://doi.org/10.5194/gmd-16-4137-2023, 2023

Short summary

Caiyi Jin et al.

Viewed

Total article views: 856 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
655	170	31	856	8	11

HTML: 655
PDF: 170
XML: 31
Total: 856
BibTeX: 8
EndNote: 11

Views and downloads (calculated since 27 Oct 2022)

Month	HTML	PDF	XML	Total
Oct 2022	145	35	12	192
Nov 2022	152	47	6	205
Dec 2022	78	16	2	96
Jan 2023	65	11	4	80
Feb 2023	61	19	0	80
Mar 2023	52	9	2	63
Apr 2023	54	6	1	61
May 2023	18	4	1	23
Jun 2023	15	4	2	21
Jul 2023	15	19	1	35

Cumulative views and downloads (calculated since 27 Oct 2022)

Month	HTML	PDF	XML	Total
Oct 2022	145	35	12	192
Nov 2022	152	47	6	205
Dec 2022	78	16	2	96
Jan 2023	65	11	4	80
Feb 2023	61	19	0	80
Mar 2023	52	9	2	63
Apr 2023	54	6	1	61
May 2023	18	4	1	23
Jun 2023	15	4	2	21
Jul 2023	15	19	1	35

Viewed (geographical distribution)

Total article views: 846 (including HTML, PDF, and XML) Thereof 846 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 24 Jul 2023

Download

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Preprint (4186 KB)
Metadata XML

Short summary

The semi-empirical physical approach derives PM_2.5 with strong physical significance. However, due to the complex optical characteristic, the physical parameters are difficult to express accurately. Thus, combining the atmospheric physical mechanism and machine learning, we propose an optimized model. It creatively embeds the random forest model into the physical PM_2.5 remote sensing approach to simulate a physical parameter. Our method shows great optimized performance in the validations.


Total:	0
HTML:	0
PDF:	0
XML:	0

An optimized semi-empirical physical approach for satellite-based PM2.5 retrieval: embedding machine learning to simulate complex physical parameters

Journal article(s) based on this preprint

Interactive discussion

Interactive discussion

Peer review completion

Suggestions for revision or reasons for rejection

Suggestions for revision or reasons for rejection

Suggestions for revision or reasons for rejection

Journal article(s) based on this preprint

Viewed

Viewed (geographical distribution)

An optimized semi-empirical physical approach for satellite-based PM_2.5 retrieval: embedding machine learning to simulate complex physical parameters