19 Jan 2024
 | 19 Jan 2024
Status: this preprint is open for discussion and under review for Atmospheric Measurement Techniques (AMT).

3-D Cloud Masking Across a Broad Swath using Multi-angle Polarimetry and Deep Learning

Sean R. Foley, Kirk D. Knobelspiesse, Andrew M. Sayer, Meng Gao, James Hays, and Judy Hoffman

Abstract. Understanding the 3-dimensional structure of clouds is of crucial importance to modeling our changing climate. Active sensors, such as radar and lidar, provide accurate vertical cloud profiles, but are mostly restricted to along-track sampling. Passive sensors can capture a wide swath, but struggle to see beneath cloud tops. In essence, both types of products are restricted to two dimensions: as a cross-section in the active case, and an image in the passive case. However, multi-angle sensor configurations contain implicit information about 3D structure, due to parallax and atmospheric path differences. Extracting that implicit information can be challenging, requiring computationally expensive radiative transfer techniques that must make limiting assumptions. Machine learning, as an alternative, may be able to capture some of the complexity of a full 3D radiative transfer solution with significantly less computational expense. In this work, we make three contributions towards understanding 3D cloud structure from multi-angle polarimetry. First, we introduce a large-scale, open-source dataset that fuses existing cloud products into a format more amenable to machine learning. This dataset treats multi-angle polarimetry as an input, and radar-based vertical cloud profiles as an output. Second, we describe and evaluate strong baseline machine learning models based that predict these profiles from the passive imagery. Notably, these models are trained only on center-swath labels, but can predict cloud profiles over the entire passive imagery swath. Third, we leverage the information-theoretic nature of machine learning to draw conclusions about the relative utility of various sensor configurations, including spectral channels, viewing angles, and polarimetry. These findings have implications for Earth-observing missions such as NASA's Plankton, Aerosol, Cloud-ocean Ecosystem (PACE) and Atmosphere Observing System (AOS) missions, as well as in informing future applications of computer vision to atmospheric remote sensing.

Sean R. Foley, Kirk D. Knobelspiesse, Andrew M. Sayer, Meng Gao, James Hays, and Judy Hoffman

Status: open (until 23 Feb 2024)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on egusphere-2023-2392', Anonymous Referee #2, 05 Feb 2024 reply
  • RC2: 'Comment on egusphere-2023-2392', Anonymous Referee #1, 06 Feb 2024 reply
Sean R. Foley, Kirk D. Knobelspiesse, Andrew M. Sayer, Meng Gao, James Hays, and Judy Hoffman

Data sets

A-Train Cloud Segmentation Dataset Sean Foley

Sean R. Foley, Kirk D. Knobelspiesse, Andrew M. Sayer, Meng Gao, James Hays, and Judy Hoffman


Total article views: 193 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
140 46 7 193 1 3
  • HTML: 140
  • PDF: 46
  • XML: 7
  • Total: 193
  • BibTeX: 1
  • EndNote: 3
Views and downloads (calculated since 19 Jan 2024)
Cumulative views and downloads (calculated since 19 Jan 2024)

Viewed (geographical distribution)

Total article views: 184 (including HTML, PDF, and XML) Thereof 184 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
Latest update: 21 Feb 2024
Short summary
Measuring the shape of clouds helps scientists understand how the Earth will continue to respond to climate change. Satellites measure clouds in different ways. One way is to take pictures of clouds from multiple angles, and to use the differences between the pictures to measure cloud structure. However, doing this accurately can be challenging. We propose a way to use machine learning to recover the shape of clouds from multi-angle satellite data.