the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 07 May 2025
Classifying Thermodynamic Cloud Phase Using Machine Learning Models
Abstract. Vertically resolved thermodynamic cloud phase classifications are essential for studies of atmospheric cloud and precipitation processes. The Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) THERMOCLDPHASE Value-Added Product (VAP) uses a multi-sensor approach to classify thermodynamic cloud phase by combining lidar backscatter and depolarization, radar reflectivity, Doppler velocity, spectral width, microwave radiometer-derived liquid water path, and radiosonde temperature measurements. The measured voxels are classified as ice, snow, mixed-phase, liquid (cloud water), drizzle, rain, and liq_driz (liquid+drizzle). We use this product as the ground truth to train three machine learning (ML) models to predict the thermodynamic cloud phase from multi-sensor remote sensing measurements taken at the ARM North Slope of Alaska (NSA) observatory: a random forest (RF), a multilayer perceptron (MLP), and a convolutional neural network (CNN) with a U-Net architecture. Evaluations against the outputs of the THERMOCLDPHASE VAP with one year of data show that the CNN outperforms the other two models, achieving the highest test accuracy, F1-score, and mean Intersection over Union (IOU). Analysis of ML confidence scores shows ice, rain, and snow have higher confidence scores, followed by liquid, while mixed, drizzle, and liq_driz have lower scores. Feature importance analysis reveals that the mean Doppler velocity and vertically resolved temperature are the most influential datastreams for ML thermodynamic cloud phase predictions. The ML models’ generalization capacity is further evaluated by applying them at another Arctic ARM site in Norway using data taken during the ARM Cold-Air Outbreaks in the Marine Boundary Layer Experiment (COMBLE) field campaign. Finally, we evaluate the ML models’ response to simulated instrument outages and signal degradation.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(2338 KB)
-
Supplement
(1704 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2338 KB) - Metadata XML
-
Supplement
(1704 KB) - BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2025-1501', Anonymous Referee #1, 27 May 2025
- AC1: 'Reply on RC1', Lexie Goldberger, 11 Jul 2025
-
RC2: 'Comment on egusphere-2025-1501', Anonymous Referee #2, 29 May 2025
Overall Comments:
Goldberger et al. provides a detailed analysis of cloud phase classification using a suite of increasingly sophisticated machine learning (ML) models. The authors find that the traditional threshold-based approach (i.e., the THERMOCLDPHASE VAP) was limited in its ability to classify phase across varying regional climates due to the use of static thresholds, and lacked a general robustness when some inputs were missing (e.g. due to instrument outages). Shifting to an ML-based approach improved classification accuracy at two locations, and provided a framework for more easily understanding variable importance and uncertainty quantification in their predictions. The paper is well structured, and includes many of the minor, yet fundamental, points (e.g.., information about data standardization, splitting, model complexity) that are often missed in these types of manuscripts/projects. I also appreciated the additional work provided in the Supplement detailing the hyperparameter search space and dropout tests. I don’t see any critical methodological issues here, the content aligns well with the journal’s mission, and I believe this paper would be of great interest to the readers of Atmospheric Measurement Techniques (AMT). However, I do have a few minor points below that I’d like to see addressed on before I fully recommend the paper for publication.
General Comments:
- While the performance of the ML models across both sites were mostly consistent, it was clear that the CNN really struggled with liquid/mixed-phase at ANX compared to NSA. Do the authors have more of a physical basis for this drop in accuracy? You discuss this issue briefly in 451-462, but I am curious if the regional cloud structures are different enough in Norway to cause the CNN to struggle for this very important classification category. On a similar note, while performance for other classes remained highest in the CNN, I found the MLP to often provide a consistent performance more generally across all classes (not only in Fig. 9, but also Fig. 6). For instance, in Fig. 6, the CNN really excels at ice-based classifications, but it is slightly worse in nearly every other class compared to the MLP/RF. Some may argue that a less complex model that is more consistent and quicker to train would be preferred compared to the CNN, even with worse performance in some categories.
- General comments about figure quality. As this is a visual project, I feel that the visual comparisons between methods are key for demonstrating performance differences to the reader. For instance, on Fig. 3, could you not zoom in more on the actual cloud structures? The plots extend to 12 km currently, but there is little-to-no activity above say 7 km (and similar for Fig. 9/10/11) which leaves a lot of blank space. Additionally, I like the point being made by Fig. 4, but it is quite challenging to read in its current state. I would make this a 4x2 plot instead of 2x4 and try to make the bars wider. I recognize that the versions of these images I am seeing is also compressed, but to highlight fine structural details, I’d recommend the authors make sure their resolution is as high as is allowed by the journal so readers can zoom in to see the interesting structural details better.
Specific Comments:
- Lines 18-19: I like the evolution from decision-tree based methods to more advanced NNs, but was curious if you had considered vision transformers for this problem? They’ve seen quite a surge in popularity over the past few years especially in areas like remote sensing (e.g., Lenhardt et al., 2024 for clouds, or Thisanke et al., 2023 more generally), and while likely beyond the scope of this project, might be of interest down the road as they can pick up on global features often missed by the U-Net.
- Lines 177: I appreciate the authors providing a list of RF hyperparameters, and I was curious about using 20 as the maximum depth here. 20 is a fairly deep tree for 100k samples and I am wondering how sensitive the model is to changing this value to something smaller (10-15).
- Line 185: Interesting, I’ve never seen someone use the scikit implementation to train an MLP (usually torch or tensorflow)! I’m guessing you used one of these for the CNN?
- Lines 202-203: How are missing pixels handled before being ingested by the CNN? Like what if you have to QC a small patch of bad radar data and mark it as missing, is that whole scene dropped?
- Figure 3: Zooming in on e-g of Fig. 3, we can see the melting layer is one of the most challenging areas to accurately predict for the CNN, with low confidence throughout. As this is an area where change is quickly happening to particle phase, shape and fallspeed, the CNN appears to struggle in resolving these fine scale details that are quite important (the presence, location, and width of the ML are often some of the most important features we would like to accurately classify for looking at regime shifts). Do you have any insights on how to improve this detection here? There have been some projects using ML to better predict this region (e.g., Brast and Markmann, 2020), but it remains an active area of research.
- Figure 8: While I think this is a useful figure, it is a bit busy with how it is currently set up and takes a moment to digest (variable by phase by model). Do you have an overall feature importance chart anywhere to give a general overview of each input’s importance across all phase types?
- Table 4: I see in Table 4 that you experiment with leaving out different variables, but did you perform retraining on these models without the combination of MPL/LWP variables which display little relevance in Fig. 8? It would appear as though they don’t provide much useful context beyond what is given in the other variables.
- Section 3.3: While not critical to the current manuscript, did you consider looking at feature/saliency maps in the CNN (e.g., Haar et al., 2023)? As a vision model, this can sometimes be useful for sanity checks that the model is looking at regions in the data that we would expect from a physical standpoint as being relevant (e.g. cloud edges, hydrometeor gradients, gaps etc.).
- Section 3.4: It is great to see the model also tested at an independent site. I touched on this in my general comment above, but how well do you believe this model could generalize to other regional climates? For instance, other DOE ARM sites like SBS/SGP/ENA or even AWR with similar instrument setups?
- Line 459: This should be CNN not CCN I believe.
- Line 602: Advanced U-Net models like these have already started to be used in recent literature at other DOE sites looking at clouds using radar data (e.g., King et al., 2025)
- Line 612: The GitHub repo link appears to be broken here. It would be great to also have a Google Colab notebook available to reproduce some of the model results if possible.
References
Brast, M. and Markmann, P.: Detecting the melting layer with a micro rain radar using a neural network approach, Atmos. Meas. Tech., 13, 6645–6656, https://doi.org/10.5194/amt-13-6645-2020, 2020.
Haar, L. V., Elvira, T., & Ochoa, O. (2023). An analysis of explainability methods for convolutional neural networks. Engineering Applications of Artificial Intelligence, 117, 105606. https://doi.org/10.1016/j.engappai.2022.105606
King, F., C. Pettersen, C. G. Fletcher, and A. Geiss, 2024: Development of a Full-Scale Connected U-Net for Reflectivity Inpainting in Spaceborne Radar Blind Zones. Artif. Intell. Earth Syst., 3, e230063, https://doi.org/10.1175/AIES-D-23-0063.1.
Lenhardt, J., Quaas, J., Sejdinovic, D., and Klocke, D.: CloudViT: classifying cloud types in global satellite data and in kilometre-resolution simulations using vision transformers, EGUsphere [preprint], https://doi.org/10.5194/egusphere-2024-2724, 2024.
Thisanke, H., Deshan, C., Chamith, K., Seneviratne, S., Vidanaarachchi, R., & Herath, D. (2023). Semantic segmentation using Vision Transformers: A survey. Engineering Applications of Artificial Intelligence, 126, 106669. https://doi.org/10.1016/j.engappai.2023.106669
Citation: https://doi.org/10.5194/egusphere-2025-1501-RC2 - AC2: 'Reply on RC2', Lexie Goldberger, 11 Jul 2025
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2025-1501', Anonymous Referee #1, 27 May 2025
This manuscript uses the U.S. Department of Energy’s THERMOCLDPHASE Value Added Product (VAP) that contains information about the vertical structure of clouds and precipitation to train three machine learning models to predict cloud thermodynamic phase at the North Slope of Alaska. The THERMOCLDPHASE VAP uses radiosondes, radar, and lidar, and a microwave radiometer to classify pixels as belonging to one of seven categories: 1) ice, 2) snow, 3) mixed-phase, 4) liquid, 5) drizzle, 6) rain, and 7) liquid + drizzle. The three machine learning models are 1) random forest (RF), 2) multi-layer perceptron (MLP) neural network, and 3) convolutional neural network (CNN) using a U-Net architecture. The authors evaluate the three machine learning models in terms of their ability to classify pixels into the seven categories using the THERMOCLDPHASE VAP as ground truth. They found that the CNN outperforms the other two models, correctly identifying ice pixels frequently, which they attribute to the fact that it takes into account the vertical structure of the pixels holistically whereas the other models consider pixels individually. They also found that Doppler velocity and vertical profiles of temperature are the most important variables for correctly predicting cloud thermodynamic phase using “feature importance analysis”. They then test the ability of their models to correctly classify the pixels into the seven categories at the COMBLE field campaign (for the period between Feb 11 – May 31, 2020) which deployed the same four instruments used in the VAP to determine how generally their models are applicable to other sites. They find that like the NSA results, the CNN outperforms the other two models and correctly identifies the frequent presence of ice pixels. Finally, the authors also trained a model (called the CNN-ICD) to test how resilient the model is to missing data. This model is based on the U-Net CNN but differs in that it includes an additional layer to drop out random input channels during training. Consistent with their “feature importance analysis”, they find that vertical temperature profiles and radar variables are the most important variables for reproducing the THERMODCLDPHASE VAP product at the NSA site.
This work addresses an important aspect of polar clouds based on the recent THERMODCLDPHASE VAP, elucidates aspects of its algorithm, and creatively investigates how to extend it using machine learning. I believe their product will be a useful contribution to the scientific community. My only main concern is the relatively minor role that lidar plays in the phase classifications and therefore the relatively poor skill that the models have in predicting liquid. I suggest that the authors delve further into how liquid phase predictions can be improved while removing the somewhat redundant parts of the manuscript describing the drop-out experiments described below. The results also seem to show that temperature plays a more dominant role than shown by the analysis methods. I would recommend publication of this manuscript after the authors consider the suggestions below.
Clarifying the geographical scope of the work
- It is unclear whether the scope of the THERMOCLDPHASE VAP is limited to Arctic (or polar) clouds. Different regions may require different tuning in the algorithm, and the authors have only focused on the Arctic (NSA and COMBLE regions) in this manuscript. If the goal is to test the generalization of the machine learning models to other regions, does this also include different cloud types? Is THERMOCLDPHASE suited for classifying the thermodynamic phase of other types of clouds? Line 57 mentions “several other ARM observatories across the world” but does not specify which. I recommend that the authors clarify this in the manuscript.
Clarifying the roles of temperature and lidar
- The sharp cut-off along the 0˚C isotherm in Figure 1(g) where ice transitions to warm precipitation suggests to me that temperature plays a critical role in determining the phase of hydrometeors. Yet, it seems that the “feature importance analysis” and Figure 8 show that radar plays an even more important role than temperature. How do the authors reconcile this?
- Also, I am concerned about the minor role that lidar plays in the phase classification presumably due to lidar attenuation. The fact that the CNN performs the best out of the three models due to its accurate prediction of ice makes sense in light of the fact that radar and temperature play the most important roles in the prediction --- radar can better observe the ice particles that are larger in size, and ice crystals that freeze homogeneously are easier to identify. I suggest that the authors separately show cases that are dominated by single-layer thin liquid clouds to check whether lidar plays a more significant role and whether the models might also show high fidelity for the liquid classifications.
Suggestions with regards to writing:
Abstract:
- The Abstract does not mention what the results for COMBLE are.
- Similarly, the results of the ML models’ response to simulated instrument outages and signal degradation are also not summarized in the Abstract.
Introduction:
- Lines 31-33 require references. Suggestions: for ice particle production (via the WBF process ---Storlevmo & Tan 2015 ), precipitation formation (Mulmenstadt et al. 2015), the evolution of the cloud life cycle (Pithan et al. 2014), and also the response of clouds to global warming (Tan et al. 2025).
- Lines 36-38: satellite remote sensing could also be included here, e.g. MODIS cloud retrievals as detailed in Platnick et al. (2016)
Concerns regarding redundancy:
- Section 4 essentially shows what the feature importance analysis did earlier in the manuscript regarding the importance of radar and the temperature soundings for the phase predictions. The authors might want to consider removing this section and replacing it with more detailed analysis on the limited role of lidar and how the classification of liquid pixels can be improved.
Minor/typographical suggestions:
- Please clarify what is meant by “pixel” and “voxel” and also be consistent with the terminology throughout.
- Line 41: “imagers” not “images”?
- Line 104: no dash necessary in “reads-in”
- Please consider including isotherms in panels (a) – (e) as well.
- Line 158: what is the percentage of “unknown” pixels in the VAP?
- Table 1: Please define “clip”.
- Lines 237-238: Why was the data limited to only 2018-2020 instead of the full record going back to 1998?
- Line 170: “imbalanced” in place of “imbalance”?
- Line 459: CCN should be CNN
- Line 252: “In the” in front of “Future”?
- Line 284: apostrophe after “models”
- Figure 5: Setting the max of the y-axis to 80 may help enhance the visibility of the drizzle/liq_driz/rain categories.
References:
Storelvmo, Trude, and Ivy Tan. "The Wegener-Bergeron-Findeisen process–Its discovery and vital importance for weather and climate." Meteorologische Zeitschrift 24.4 (2015): 455-461.
Mülmenstädt, Johannes, et al. "Frequency of occurrence of rain from liquid‐, mixed‐, and ice‐phase clouds derived from A‐Train satellite retrievals." Geophysical Research Letters 42.15 (2015): 6502-6509.
Pithan, Felix, Brian Medeiros, and Thorsten Mauritsen. "Mixed-phase clouds cause climate model biases in Arctic wintertime temperature inversions." Climate dynamics 43 (2014): 289-303.
Tan, Ivy, et al. "Moderate climate sensitivity due to opposing mixed-phase cloud feedbacks." npj Climate and Atmospheric Science 8.1 (2025): 86.
Platnick, Steven, et al. "The MODIS cloud optical and microphysical products: Collection 6 updates and examples from Terra and Aqua." IEEE Transactions on Geoscience and Remote Sensing 55.1 (2016): 502-525.
Citation: https://doi.org/10.5194/egusphere-2025-1501-RC1 - AC1: 'Reply on RC1', Lexie Goldberger, 11 Jul 2025
-
RC2: 'Comment on egusphere-2025-1501', Anonymous Referee #2, 29 May 2025
Overall Comments:
Goldberger et al. provides a detailed analysis of cloud phase classification using a suite of increasingly sophisticated machine learning (ML) models. The authors find that the traditional threshold-based approach (i.e., the THERMOCLDPHASE VAP) was limited in its ability to classify phase across varying regional climates due to the use of static thresholds, and lacked a general robustness when some inputs were missing (e.g. due to instrument outages). Shifting to an ML-based approach improved classification accuracy at two locations, and provided a framework for more easily understanding variable importance and uncertainty quantification in their predictions. The paper is well structured, and includes many of the minor, yet fundamental, points (e.g.., information about data standardization, splitting, model complexity) that are often missed in these types of manuscripts/projects. I also appreciated the additional work provided in the Supplement detailing the hyperparameter search space and dropout tests. I don’t see any critical methodological issues here, the content aligns well with the journal’s mission, and I believe this paper would be of great interest to the readers of Atmospheric Measurement Techniques (AMT). However, I do have a few minor points below that I’d like to see addressed on before I fully recommend the paper for publication.
General Comments:
- While the performance of the ML models across both sites were mostly consistent, it was clear that the CNN really struggled with liquid/mixed-phase at ANX compared to NSA. Do the authors have more of a physical basis for this drop in accuracy? You discuss this issue briefly in 451-462, but I am curious if the regional cloud structures are different enough in Norway to cause the CNN to struggle for this very important classification category. On a similar note, while performance for other classes remained highest in the CNN, I found the MLP to often provide a consistent performance more generally across all classes (not only in Fig. 9, but also Fig. 6). For instance, in Fig. 6, the CNN really excels at ice-based classifications, but it is slightly worse in nearly every other class compared to the MLP/RF. Some may argue that a less complex model that is more consistent and quicker to train would be preferred compared to the CNN, even with worse performance in some categories.
- General comments about figure quality. As this is a visual project, I feel that the visual comparisons between methods are key for demonstrating performance differences to the reader. For instance, on Fig. 3, could you not zoom in more on the actual cloud structures? The plots extend to 12 km currently, but there is little-to-no activity above say 7 km (and similar for Fig. 9/10/11) which leaves a lot of blank space. Additionally, I like the point being made by Fig. 4, but it is quite challenging to read in its current state. I would make this a 4x2 plot instead of 2x4 and try to make the bars wider. I recognize that the versions of these images I am seeing is also compressed, but to highlight fine structural details, I’d recommend the authors make sure their resolution is as high as is allowed by the journal so readers can zoom in to see the interesting structural details better.
Specific Comments:
- Lines 18-19: I like the evolution from decision-tree based methods to more advanced NNs, but was curious if you had considered vision transformers for this problem? They’ve seen quite a surge in popularity over the past few years especially in areas like remote sensing (e.g., Lenhardt et al., 2024 for clouds, or Thisanke et al., 2023 more generally), and while likely beyond the scope of this project, might be of interest down the road as they can pick up on global features often missed by the U-Net.
- Lines 177: I appreciate the authors providing a list of RF hyperparameters, and I was curious about using 20 as the maximum depth here. 20 is a fairly deep tree for 100k samples and I am wondering how sensitive the model is to changing this value to something smaller (10-15).
- Line 185: Interesting, I’ve never seen someone use the scikit implementation to train an MLP (usually torch or tensorflow)! I’m guessing you used one of these for the CNN?
- Lines 202-203: How are missing pixels handled before being ingested by the CNN? Like what if you have to QC a small patch of bad radar data and mark it as missing, is that whole scene dropped?
- Figure 3: Zooming in on e-g of Fig. 3, we can see the melting layer is one of the most challenging areas to accurately predict for the CNN, with low confidence throughout. As this is an area where change is quickly happening to particle phase, shape and fallspeed, the CNN appears to struggle in resolving these fine scale details that are quite important (the presence, location, and width of the ML are often some of the most important features we would like to accurately classify for looking at regime shifts). Do you have any insights on how to improve this detection here? There have been some projects using ML to better predict this region (e.g., Brast and Markmann, 2020), but it remains an active area of research.
- Figure 8: While I think this is a useful figure, it is a bit busy with how it is currently set up and takes a moment to digest (variable by phase by model). Do you have an overall feature importance chart anywhere to give a general overview of each input’s importance across all phase types?
- Table 4: I see in Table 4 that you experiment with leaving out different variables, but did you perform retraining on these models without the combination of MPL/LWP variables which display little relevance in Fig. 8? It would appear as though they don’t provide much useful context beyond what is given in the other variables.
- Section 3.3: While not critical to the current manuscript, did you consider looking at feature/saliency maps in the CNN (e.g., Haar et al., 2023)? As a vision model, this can sometimes be useful for sanity checks that the model is looking at regions in the data that we would expect from a physical standpoint as being relevant (e.g. cloud edges, hydrometeor gradients, gaps etc.).
- Section 3.4: It is great to see the model also tested at an independent site. I touched on this in my general comment above, but how well do you believe this model could generalize to other regional climates? For instance, other DOE ARM sites like SBS/SGP/ENA or even AWR with similar instrument setups?
- Line 459: This should be CNN not CCN I believe.
- Line 602: Advanced U-Net models like these have already started to be used in recent literature at other DOE sites looking at clouds using radar data (e.g., King et al., 2025)
- Line 612: The GitHub repo link appears to be broken here. It would be great to also have a Google Colab notebook available to reproduce some of the model results if possible.
References
Brast, M. and Markmann, P.: Detecting the melting layer with a micro rain radar using a neural network approach, Atmos. Meas. Tech., 13, 6645–6656, https://doi.org/10.5194/amt-13-6645-2020, 2020.
Haar, L. V., Elvira, T., & Ochoa, O. (2023). An analysis of explainability methods for convolutional neural networks. Engineering Applications of Artificial Intelligence, 117, 105606. https://doi.org/10.1016/j.engappai.2022.105606
King, F., C. Pettersen, C. G. Fletcher, and A. Geiss, 2024: Development of a Full-Scale Connected U-Net for Reflectivity Inpainting in Spaceborne Radar Blind Zones. Artif. Intell. Earth Syst., 3, e230063, https://doi.org/10.1175/AIES-D-23-0063.1.
Lenhardt, J., Quaas, J., Sejdinovic, D., and Klocke, D.: CloudViT: classifying cloud types in global satellite data and in kilometre-resolution simulations using vision transformers, EGUsphere [preprint], https://doi.org/10.5194/egusphere-2024-2724, 2024.
Thisanke, H., Deshan, C., Chamith, K., Seneviratne, S., Vidanaarachchi, R., & Herath, D. (2023). Semantic segmentation using Vision Transformers: A survey. Engineering Applications of Artificial Intelligence, 126, 106669. https://doi.org/10.1016/j.engappai.2023.106669
Citation: https://doi.org/10.5194/egusphere-2025-1501-RC2 - AC2: 'Reply on RC2', Lexie Goldberger, 11 Jul 2025
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 805 | 79 | 23 | 907 | 33 | 29 | 48 |
- HTML: 805
- PDF: 79
- XML: 23
- Total: 907
- Supplement: 33
- BibTeX: 29
- EndNote: 48
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Lexie Goldberger
Maxwell Levin
Carlandra Harris
Andrew Geiss
Matthew D. Shupe
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2338 KB) - Metadata XML
-
Supplement
(1704 KB) - BibTeX
- EndNote
- Final revised paper
This manuscript uses the U.S. Department of Energy’s THERMOCLDPHASE Value Added Product (VAP) that contains information about the vertical structure of clouds and precipitation to train three machine learning models to predict cloud thermodynamic phase at the North Slope of Alaska. The THERMOCLDPHASE VAP uses radiosondes, radar, and lidar, and a microwave radiometer to classify pixels as belonging to one of seven categories: 1) ice, 2) snow, 3) mixed-phase, 4) liquid, 5) drizzle, 6) rain, and 7) liquid + drizzle. The three machine learning models are 1) random forest (RF), 2) multi-layer perceptron (MLP) neural network, and 3) convolutional neural network (CNN) using a U-Net architecture. The authors evaluate the three machine learning models in terms of their ability to classify pixels into the seven categories using the THERMOCLDPHASE VAP as ground truth. They found that the CNN outperforms the other two models, correctly identifying ice pixels frequently, which they attribute to the fact that it takes into account the vertical structure of the pixels holistically whereas the other models consider pixels individually. They also found that Doppler velocity and vertical profiles of temperature are the most important variables for correctly predicting cloud thermodynamic phase using “feature importance analysis”. They then test the ability of their models to correctly classify the pixels into the seven categories at the COMBLE field campaign (for the period between Feb 11 – May 31, 2020) which deployed the same four instruments used in the VAP to determine how generally their models are applicable to other sites. They find that like the NSA results, the CNN outperforms the other two models and correctly identifies the frequent presence of ice pixels. Finally, the authors also trained a model (called the CNN-ICD) to test how resilient the model is to missing data. This model is based on the U-Net CNN but differs in that it includes an additional layer to drop out random input channels during training. Consistent with their “feature importance analysis”, they find that vertical temperature profiles and radar variables are the most important variables for reproducing the THERMODCLDPHASE VAP product at the NSA site.
This work addresses an important aspect of polar clouds based on the recent THERMODCLDPHASE VAP, elucidates aspects of its algorithm, and creatively investigates how to extend it using machine learning. I believe their product will be a useful contribution to the scientific community. My only main concern is the relatively minor role that lidar plays in the phase classifications and therefore the relatively poor skill that the models have in predicting liquid. I suggest that the authors delve further into how liquid phase predictions can be improved while removing the somewhat redundant parts of the manuscript describing the drop-out experiments described below. The results also seem to show that temperature plays a more dominant role than shown by the analysis methods. I would recommend publication of this manuscript after the authors consider the suggestions below.
Clarifying the geographical scope of the work
Clarifying the roles of temperature and lidar
Suggestions with regards to writing:
Abstract:
Introduction:
Concerns regarding redundancy:
Minor/typographical suggestions:
References:
Storelvmo, Trude, and Ivy Tan. "The Wegener-Bergeron-Findeisen process–Its discovery and vital importance for weather and climate." Meteorologische Zeitschrift 24.4 (2015): 455-461.
Mülmenstädt, Johannes, et al. "Frequency of occurrence of rain from liquid‐, mixed‐, and ice‐phase clouds derived from A‐Train satellite retrievals." Geophysical Research Letters 42.15 (2015): 6502-6509.
Pithan, Felix, Brian Medeiros, and Thorsten Mauritsen. "Mixed-phase clouds cause climate model biases in Arctic wintertime temperature inversions." Climate dynamics 43 (2014): 289-303.
Tan, Ivy, et al. "Moderate climate sensitivity due to opposing mixed-phase cloud feedbacks." npj Climate and Atmospheric Science 8.1 (2025): 86.
Platnick, Steven, et al. "The MODIS cloud optical and microphysical products: Collection 6 updates and examples from Terra and Aqua." IEEE Transactions on Geoscience and Remote Sensing 55.1 (2016): 502-525.