Analysis of cloud fraction adjustment to aerosols and its dependence on meteorological controls using explainable machine learning

Jia, Yichen; Andersen, Hendrik; Cermak, Jan

doi:https://doi.org/10.5194/egusphere-2023-1667

Preprints

https://doi.org/10.5194/egusphere-2023-1667

Preprints

08 Aug 2023

| 08 Aug 2023

Analysis of cloud fraction adjustment to aerosols and its dependence on meteorological controls using explainable machine learning

Yichen Jia, Hendrik Andersen, and Jan Cermak

Abstract. Aerosol-cloud interactions (ACI) have a pronounced influence on the Earth’s radiation budget but continue to pose one of the most substantial uncertainties in the climate system. Marine boundary-layer clouds (MBLCs) are particularly important since they cover a large portion of the Earth’s surface. One of the biggest challenges in quantifying ACI from observations lies in isolating adjustments of cloud fraction (CLF) to aerosol perturbations from the covariability and influence of the local meteorological conditions. In this study, this isolation is attempted using nine years (2011–2019) of near-global daily satellite cloud products in combination with reanalysis data of meteorological parameters. With cloud-droplet number concentration (N_d) as a proxy for aerosol, MBLC CLF is predicted by region-specific gradient boosting machine learning models. By means of SHapley Additive exPlanation (SHAP) regression values, CLF sensitivity to N_d and meteorological factors as well as meteorological influences on the N_d–CLF sensitivity are quantified. The regional ML models are able to capture on average 45 % of the CLF variability. Global patterns of CLF sensitivity show that CLF is positively associated with N_d, in particular in the stratocumulus-to-cumulus transition regions and in the Southern Ocean. CLF sensitivity to estimated inversion strength (EIS) is ubiquitously positive and strongest in tropical and subtropical regions topped by stratocumulus and within the midlatitudes. Globally, increased sea surface temperature (SST) reduces CLF, particularly in stratocumulus regions. The spatial patterns of CLF sensitivity to horizontal wind components in the free troposphere point to the impact of synoptic-scale weather systems and vertical wind shear on MBLCs. The N_d–CLF relationship is found to depend more on the selected thermodynamical variables than dynamical variables, and in particular on EIS and SST. In the midlatitudes, a stronger inversion is found to amplify the N_d–CLF relationship, while this is not observed in the stratocumulus regions. In the stratocumulus-to-cumulus transition regions, the N_d–CLF sensitivity is found to be amplified by higher SSTs, potentially pointing to N_d more frequently delaying this transition in these conditions. The expected climatic changes of EIS and SST may thus influence future forcings from ACIs. The near-global ML framework introduced in this study produces a better quantification of the response of MBLC CLF to aerosols taking into account the covariations with meteorology.

Received: 20 Jul 2023 – Discussion started: 08 Aug 2023

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.

Download & links

Preprint (PDF, 2152 KB)

Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
Preprint (2152 KB)

Download & links

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Journal article(s) based on this preprint

26 Nov 2024

Analysis of the cloud fraction adjustment to aerosols and its dependence on meteorological controls using explainable machine learning

Yichen Jia, Hendrik Andersen, and Jan Cermak

Atmos. Chem. Phys., 24, 13025–13045, https://doi.org/10.5194/acp-24-13025-2024,https://doi.org/10.5194/acp-24-13025-2024, 2024

Short summary

Yichen Jia, Hendrik Andersen, and Jan Cermak

Interactive discussion

Status: closed

RC1: 'Comment on egusphere-2023-1667', Anonymous Referee #1, 29 Aug 2023

This is an interesting work focused on factors influencing the cloud fraction by using explainable machine learning approaches. The data and method are both solid, and the paper is well-written. I just found several places that need to be justified or clarified. Therefore, I recommend a minor revision for this paper to be published on ACP.
Line 90: Does re means CLF? How cloud temperature, solar zenith viewing angle, and satellite zenith angle are used to filter and compute Nd?

Line 110: How the reanalysis data are harmonized? Could you please provide a little more details, which spatial averaging (interpolation) techniques have been used?

Line 114: Why use 99 percentiles as the threshold? How extreme values will influence your interpretation of the ML model?

Line 136: 6000 data points as the threshold, would you please provide an estimate for the percentage of data dropped from the total sample?

Line 180: The definition of IAI may need to be clarified, is it a slope or difference? how the difference is calculated, which minus which?

Line 214: How do you explain that the PF is influencing the CLF but not vice versa? Same for some other predictors in the ML model.

Line 229, SHF, please declare the full name of the acronym when it first appears in the text, even if it has been listed in table 1.

Section 3.3.2, Fig. 8, why RH850 is omitted? It shows higher importance than SST in Fig.7.

Citation: https://doi.org/10.5194/egusphere-2023-1667-RC1
RC2: 'Comment on egusphere-2023-1667', Anonymous Referee #2, 30 Sep 2023

This manuscript fits xgboost models to capture daily cloud fraction at 5x5 degree scale. They apply the SHAP calculation and present results mostly in terms of the SHAP values and their variations to different values by holding individual variables out from model calculations. The authors argue that the results represent a 'better quantification of the responses of MBLC CLF to aerosols'. The topic is relevant for ACP. However, there are a few major methodological concerns I have and they are detailed in the following. In my opinion, they must be addressed before the paper can be published.
1. It is unclear what data constitute MBL CLF. Only daily MODIS level-3 data are mentioned, but AFAIK, MODIS daily data do not have a field called MBL CLF. The authors need to be clear on this.
2. It is unclear how the data is standardized. The standardization procedure is easy to understand, but is this done globally or locally for each 5x5 box?
3. Some inappropriate reference citing is noted. For example, line 37, 'Furthermore, MBLCs are especially susceptible to aerosol perturbations due to their physical properties (Wood et al., 2015).' This is a very vague and general statement with a field campaign overview paper as a reference. As a researcher working on this topic for a long time, I cannot follow what is being said or cited here. Line 45: these two references are relevant, but they are neither the earliest nor the best papers that explain the mechanism mentioned here. Line 50: Yuan et al., 2011 showed increase of cloud fraction with aerosols in the trade cumulus region. Line 52: observational evidence of GCM overestimation of LWP adjustment was presented convincingly in Toll et al., 2019. Line 102: potential retrieval biases are extensively discussed in Zhang et al., 2011. Grosvenor et al 2018 has relevant info, but it is not specifically for this subject.
4. SHAP values, as the authors corrected noted, are only ONE way of attempting to explain the boosted tree models. For each data point, there is a SHAP value for each explaining variable. By construct, they are 'situationally' dependent. They don't really provide any physical insights. All the algorithm is trying to do is gradient boosting its model to best fit the data. In fact, the first figure shows that the way the authors try to use SHAP values to 'explain' results is not physical. Figure 1a shows that SHAP value for Nd generally gets larger with increasing Nd. Physically, it says that when clouds are more polluted, cloud fraction tend to increase with Nd more stronger. This runs against our physical understanding. Fitting a slope for the SHAP values and claiming this shows sensitivity of CLF to Nd are not valid IMO. I'd love to hear the authors' rationale here.
5. What follows in the manuscript is thus questionable. I will reserve my comments for the next version after the authors address the important methodological question.

Citation: https://doi.org/10.5194/egusphere-2023-1667-RC2
AC1: 'Comment on egusphere-2023-1667', Yichen Jia, 27 Oct 2023

The authors' comments are in the uploaded pdf file.

Citation: https://doi.org/10.5194/egusphere-2023-1667-AC1

Interactive discussion

Status: closed

RC1: 'Comment on egusphere-2023-1667', Anonymous Referee #1, 29 Aug 2023

This is an interesting work focused on factors influencing the cloud fraction by using explainable machine learning approaches. The data and method are both solid, and the paper is well-written. I just found several places that need to be justified or clarified. Therefore, I recommend a minor revision for this paper to be published on ACP.
Line 90: Does re means CLF? How cloud temperature, solar zenith viewing angle, and satellite zenith angle are used to filter and compute Nd?

Line 110: How the reanalysis data are harmonized? Could you please provide a little more details, which spatial averaging (interpolation) techniques have been used?

Line 114: Why use 99 percentiles as the threshold? How extreme values will influence your interpretation of the ML model?

Line 136: 6000 data points as the threshold, would you please provide an estimate for the percentage of data dropped from the total sample?

Line 180: The definition of IAI may need to be clarified, is it a slope or difference? how the difference is calculated, which minus which?

Line 214: How do you explain that the PF is influencing the CLF but not vice versa? Same for some other predictors in the ML model.

Line 229, SHF, please declare the full name of the acronym when it first appears in the text, even if it has been listed in table 1.

Section 3.3.2, Fig. 8, why RH850 is omitted? It shows higher importance than SST in Fig.7.

Citation: https://doi.org/10.5194/egusphere-2023-1667-RC1
RC2: 'Comment on egusphere-2023-1667', Anonymous Referee #2, 30 Sep 2023

This manuscript fits xgboost models to capture daily cloud fraction at 5x5 degree scale. They apply the SHAP calculation and present results mostly in terms of the SHAP values and their variations to different values by holding individual variables out from model calculations. The authors argue that the results represent a 'better quantification of the responses of MBLC CLF to aerosols'. The topic is relevant for ACP. However, there are a few major methodological concerns I have and they are detailed in the following. In my opinion, they must be addressed before the paper can be published.
1. It is unclear what data constitute MBL CLF. Only daily MODIS level-3 data are mentioned, but AFAIK, MODIS daily data do not have a field called MBL CLF. The authors need to be clear on this.
2. It is unclear how the data is standardized. The standardization procedure is easy to understand, but is this done globally or locally for each 5x5 box?
3. Some inappropriate reference citing is noted. For example, line 37, 'Furthermore, MBLCs are especially susceptible to aerosol perturbations due to their physical properties (Wood et al., 2015).' This is a very vague and general statement with a field campaign overview paper as a reference. As a researcher working on this topic for a long time, I cannot follow what is being said or cited here. Line 45: these two references are relevant, but they are neither the earliest nor the best papers that explain the mechanism mentioned here. Line 50: Yuan et al., 2011 showed increase of cloud fraction with aerosols in the trade cumulus region. Line 52: observational evidence of GCM overestimation of LWP adjustment was presented convincingly in Toll et al., 2019. Line 102: potential retrieval biases are extensively discussed in Zhang et al., 2011. Grosvenor et al 2018 has relevant info, but it is not specifically for this subject.
4. SHAP values, as the authors corrected noted, are only ONE way of attempting to explain the boosted tree models. For each data point, there is a SHAP value for each explaining variable. By construct, they are 'situationally' dependent. They don't really provide any physical insights. All the algorithm is trying to do is gradient boosting its model to best fit the data. In fact, the first figure shows that the way the authors try to use SHAP values to 'explain' results is not physical. Figure 1a shows that SHAP value for Nd generally gets larger with increasing Nd. Physically, it says that when clouds are more polluted, cloud fraction tend to increase with Nd more stronger. This runs against our physical understanding. Fitting a slope for the SHAP values and claiming this shows sensitivity of CLF to Nd are not valid IMO. I'd love to hear the authors' rationale here.
5. What follows in the manuscript is thus questionable. I will reserve my comments for the next version after the authors address the important methodological question.

Citation: https://doi.org/10.5194/egusphere-2023-1667-RC2
AC1: 'Comment on egusphere-2023-1667', Yichen Jia, 27 Oct 2023

The authors' comments are in the uploaded pdf file.

Citation: https://doi.org/10.5194/egusphere-2023-1667-AC1

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision | EF: Editorial file upload

AR by Yichen Jia on behalf of the Authors (27 Oct 2023) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (27 Oct 2023) by Yuan Wang

RR by Anonymous Referee #1 (17 Nov 2023)

RR by Anonymous Referee #2 (06 Dec 2023)

Suggestions for revision or reasons for rejection

On the point of justifying SHAP values of a complicated tree model, the authors make the point that basically everything statistical modeling technique is doing similar things. While that to some extent is true, it is besides the point. The point is that using extremely complicated models, that is beyond say individual scientists' grasp, to 'understand' behavior of low clouds is questionable. The understanding in this context comes from calculating SHAP values of a highly nonlinear model. While SHAP values are popular, they have their own limitations: the calculation itself depends on the software package, it assumes feature independence, and most importantly these calculations are not meant for causal inference. That is precisely I made the point of checking the 'physical sense' of model behavior reported here. I used figure 1a as an example. As Nd increases, the SHAP values continue to scale with Nd (or lnNd). In other words, lnNd contributes to increase low cloud fraction linearly. This is not physical and not what we observe in nature. If anything, multiple lines of evidence suggest that Nd tend to saturate after a rather low threshold (Rosenfeld, et al., 2012; 2019; Yuan et al., 2023). This behavior is determined by low cloud precipitation formation and the threshold is quantified to be around 60-80 per CC. Similarly, I mentioned systematic biases associated with Nd calculations and low cloud fraction. This is also physical and built in with the data the authors used. In other words, there is a strong component of covariance between these two that does not have physical implications but artificial correlation. These are first order considerations when attempting to 'explain' low cloud fractions. The standardization procedure is also puzzling since it means that all the quantities calculated here are based on local data distribution and thus not really comparable between different 5x5 grids.
It is however fine for the authors to try to analyze the data and report their statistical findings. It may be a useful endeavor for readers of this paper. However, it is clear to me that the authors do not seem to have a solid background in the physical understanding of low cloud formation and aerosol effects on them while attempting to extract physical explanations from statistical models. To this point, there is a saying by Einstein that says 'Everything should be made as simple as possible, but not simpler'. The authors complied a long list of variables and used them to best fit the data. I suggest the authors to write the paper based on actual data and do not make too much into causal inference and discuss physical implications because the tool used here does not allow that. Particularly, the results do not always make physical sense anyway.

Hide

ED: Reconsider after major revisions (07 Dec 2023) by Yuan Wang

AR by Yichen Jia on behalf of the Authors (17 Jan 2024) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (20 Jan 2024) by Yuan Wang

RR by Anonymous Referee #2 (16 Feb 2024)

ED: Reconsider after major revisions (27 Feb 2024) by Yuan Wang

AR by Yichen Jia on behalf of the Authors (11 Mar 2024) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (19 Mar 2024) by Yuan Wang

RR by Anonymous Referee #3 (12 Apr 2024)

Suggestions for revision or reasons for rejection

In this work, the authors use a novel method to identify the sensitivity of cloud fraction to aerosol variations. They find that Nd is strongly correlated to CF, after accounting for the impact of other cloud controlling factors. This work is clearly in scope for ACP and would be of interest to its readers. The work is of a good standard and I think would be suitable for publication with a few additions/changes.

I appreciate the authors have already done a considerable amount of work responding to other reviewers, I would only have a few small points to add here.

It seems that the interpretation of the SHAP values are not straightforward. This is not a fault of the author or the reader, but given this method is still fairly new, a little extra explanation could be useful. I am not sure directing the reader to read a textbook targeted at computer scientists is useful - with a small amount of extra text, this paper could provide an explanation of the SHAP-based techniques accessible at a broader range of atmospheric scientists and help encourage the use of this technique.

As I undersatnd it, the SHAP value does not represent a sensitivity (as the authors say),. However, this was not immediately clear to me (having not looked at this in much detail before). Phrases like 'way to measure the relative contributions of Nd (as a surrogate for aerosols) and meteorological factors to CLF changes' initially suggested to me that we were looking at a sensitivity-like measure. Assuming I am understanding this correctly, the 'base value' here is effectively the climatological CLF at a given location (what would be predicted with no extra information. The SHAP values of Nd then show the CLF that would be predicted, given that the Nd is known (or at least the difference from the climatological CLF), such that CLF|Nd = SHAP(Nd)*sigma_CLF+CLF_clim. To me, this framing then makes it clearer that a more traditional sensitivity (dCF/dlnNd) would have to be calculated using dSHAP(Nd)/dlnNd and CLF scaling.

I am a little concerned by the very linear relationship in log space shown in Fig. 1b. Given CLF is capped at 0 and 100%, many previous studies have found a non-linear relationship even in lnNd-space (e.g. Gryspeerdt et al, JGR, 2016). It is not clear to me how such a linear relationship here can be achieved, especially when using instantaneous daily cloud property data?

There are good arguments for using the normalised values, particularly when comparing the different controls against each other. However, it is also important to be able to compare the results of this work against other studies, where the use of normalised values are less common. It would be good to have either a beta value that can be compared to the range in Bellouin et al., Rev. Geophys. (2020), or a forcing estimate that could be compared to those from other studies (e.g. Gryspeerdt et al, ACP, 2020).

I also found the mention/discussion of the potential non-causal Nd-CF link to be a bit lacking, especially around the headline results in the abstract and conclusions. While it is mentioned that there are potential retrieval biases in Nd as a function of CF, the impact of these is somewhat downplayed in the interpretation of the results. There are significant uncertainties in the retrieval of Nd in broken-cloud regimes, exactly where the observed Nd-CF relationship is strongest. While these results are clearly consistent with the theoretical behaviour of stratocumulus clouds, statements that CLF is 'sensitive to' Nd and that aerosol has a 'considerable impact on MBL cloudiness' might be over-attributing the causality of the results. This doesn't require a large change in the paper, just a little bit of rewording of some of the results.

There are a few other minor suggestion
- Is the temporal split of train-test data suitable, given the underlying change in climate? I doubt it would affect the results much, but could be something to consider for future work.

- Are the Nd values calculated from the 1x1 degree tau-re values, or from the underlying joint histograms? The use of the large-scale mean values may lead to biases, due to the non-linearity of the Nd calculation. If it is useful, there is pre-filtered Nd data available for the period of study as a companion to Gryspeerdt et al., ACP, 2022.

Hide

ED: Reconsider after major revisions (21 Apr 2024) by Yuan Wang

AR by Yichen Jia on behalf of the Authors (06 Aug 2024) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (13 Aug 2024) by Yuan Wang

RR by Anonymous Referee #3 (19 Aug 2024)

ED: Publish subject to minor revisions (review by editor) (19 Aug 2024) by Yuan Wang

AR by Yichen Jia on behalf of the Authors (27 Aug 2024) Author's response Author's tracked changes Manuscript

ED: Publish as is (10 Sep 2024) by Yuan Wang

AR by Yichen Jia on behalf of the Authors (16 Sep 2024) Manuscript

Journal article(s) based on this preprint

26 Nov 2024

Analysis of the cloud fraction adjustment to aerosols and its dependence on meteorological controls using explainable machine learning

Yichen Jia, Hendrik Andersen, and Jan Cermak

Atmos. Chem. Phys., 24, 13025–13045, https://doi.org/10.5194/acp-24-13025-2024,https://doi.org/10.5194/acp-24-13025-2024, 2024

Short summary

Yichen Jia, Hendrik Andersen, and Jan Cermak

Viewed

Total article views: 913 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
617	258	38	913	31	29

HTML: 617
PDF: 258
XML: 38
Total: 913
BibTeX: 31
EndNote: 29

Views and downloads (calculated since 08 Aug 2023)

Month	HTML	PDF	XML	Total
Aug 2023	137	64	8	209
Sep 2023	55	31	2	88
Oct 2023	42	30	6	78
Nov 2023	15	13	1	29
Dec 2023	21	11	3	35
Jan 2024	19	7	2	28
Feb 2024	23	15	1	39
Mar 2024	35	12	1	48
Apr 2024	36	22	5	63
May 2024	33	13	0	46
Jun 2024	50	12	5	67
Jul 2024	29	7	3	39
Aug 2024	36	3	0	39
Sep 2024	27	6	1	34
Oct 2024	27	6	0	33
Nov 2024	32	6	0	38

Cumulative views and downloads (calculated since 08 Aug 2023)

Month	HTML	PDF	XML	Total
Aug 2023	137	64	8	209
Sep 2023	55	31	2	88
Oct 2023	42	30	6	78
Nov 2023	15	13	1	29
Dec 2023	21	11	3	35
Jan 2024	19	7	2	28
Feb 2024	23	15	1	39
Mar 2024	35	12	1	48
Apr 2024	36	22	5	63
May 2024	33	13	0	46
Jun 2024	50	12	5	67
Jul 2024	29	7	3	39
Aug 2024	36	3	0	39
Sep 2024	27	6	1	34
Oct 2024	27	6	0	33
Nov 2024	32	6	0	38

Viewed (geographical distribution)

Total article views: 903 (including HTML, PDF, and XML) Thereof 903 with geography defined and 0 with unknown origin.

Country	#	Views	%

Cited

Latest update: 26 Nov 2024

Download

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Preprint (2152 KB)
Metadata XML

Short summary

We present a near-global observation-based explainable machine-learning framework to quantify the response of cloud fraction (CLF) of marine low clouds to cloud droplet number concentration (N_d) considering the covariations with meteorological factors. The CLF sensitivity to N_d and numerous meteorological factors as well as the dependence of the N_d–CLF sensitivity on the meteorology reveal the underlying physical mechanisms, providing a novel approach to constrain aerosol-cloud interactions.


Total:	0
HTML:	0
PDF:	0
XML:	0

Analysis of cloud fraction adjustment to aerosols and its dependence on meteorological controls using explainable machine learning

Journal article(s) based on this preprint

Interactive discussion

Interactive discussion

Peer review completion

Suggestions for revision or reasons for rejection

Suggestions for revision or reasons for rejection

Suggestions for revision or reasons for rejection

Suggestions for revision or reasons for rejection

Suggestions for revision or reasons for rejection

Journal article(s) based on this preprint

Viewed

Viewed (geographical distribution)

Cited

1 citations as recorded by crossref.