Assessing the limits of lidar aerosol inversions using a probabilistic machine learning approach
Abstract. Aerosol-cloud interactions represent a major source of uncertainty in climate models and addressing this requires accurate, vertically-resolved observations of aerosol microphysical properties such as effective radius and number concentration. While Raman and High Spectral Resolution lidars provide vital bulk optical measurements, fundamentally they do not measure these quantities directly; retrieving microphysical properties from these observations is a mathematically ill-posed and non-unique inverse problem. Existing retrieval algorithms—including traditional physics-based inversions, semi-empirical parametrization methods, and recent empirical machine learning approaches—typically force a deterministic, single-point solution. This masks fundamental uncertainties and relies on opaque priors. In this study, we introduce a machine learning framework that embraces the inherent non-uniqueness of the lidar inversions. Trained on a dataset of in situ aerosol size distributions coupled with Mie theory, our model predicts the Probability Density Function (PDF) of possible aerosol microphysical states rather than a single estimate. We leverage this framework to evaluate the theoretical capabilities and error sensitivities of four distinct lidar architectures, ranging from advanced multi-wavelength Raman configurations (6β+2α, 3β+2α) to spaceborne-relevant systems (1β+1α at 355 nm, 3β). Under idealized, low-noise conditions, advanced multi-wavelength systems best constrain the microphysical solution space. However, under realistic observational uncertainties, the accuracy of these complex architectures degrades, whereas simpler configurations with direct extinction measurements (e.g., 1β+1α) prove significantly more robust. Ultimately, this best-case analysis establishes that lidar has the potential to constrain effective radius to approximately a factor of 2 and concentration to an order of magnitude, demonstrating that probabilistic retrievals can play an important role in providing mathematically transparent observational constraints for global climate models.
Dear Matthew,
Here, some comments from my side. The aim of my comments is simply to broaden a bit the discussion towards the ‘modern’ needs of the climate and atmospheric research community. The environmental and atmospheric research communities need aerosol-type-dependent information (in the profile) on particle mass concentrations (such as PM2.5 or PM10), and especially regarding CCN (cloud condensation nucleus) and INP (ice-nucleating particle) concentrations. To provide this aerosol-type-dependent information, a separation of dust and non-dust aerosol components is required.
The manuscript shows an alternative approach (to the classical lidar inversion technique) to obtain microphysical aerosol products from multiwavelength lidar observations of backscatter and extinction coefficients. Machine learning (ML) procedures are used. Ok, good!
Discussed are the results for the retrieved effective radius and the number concentration of the aerosol particles.
Now to my points: Your study deals with the basic level of retrievals and products. The next higher level would be to separate dust from non-dust aerosol contributions, in the first step, by using the particle linear depolarization ratio, and then, in the second step, apply your ML approach to the non-dust aerosol fraction and another ML approach (for spheroidal particles) to the dust fraction.
The motivation for this separation is given by the fact that dust plays a unique role among all aerosol types when it comes to aerosol-cloud interaction. Pure dust particles are usually bad CCN because they are hydrophobic, and, in contrast, contaminated (polluted) dust are super efficient CCN because they are large and hygroscopic. Furthermore, dust particles are rather efficient INPs. Thus, there is clear need to identify and quantify the dust properties.
To my opinion, modern aerosol retrieval methods must distinguish dust and non-dust components, and, if possible deliver even a clear aerosol type information for the non dust aerosol (marine, urban, smoke, dust, volcanic aerosol by using aerosol models, MERRA, CAMS and others). All this is considered in latest aerosol methods developed for the EARTHCARE lidar data analysis.
Regarding your comments to the POLIPHON method. POLIPHON is based on the analysis of climatologically mean aerosol size distributions and climatological mean optical properties (column integrated extinction coefficients). The AERONET data base covers meanwhile observations over several of decades, at very aerosol-contrasting stations, and for all relevant aerosol types (marine, continental, dust, wildfires moke particles). Therefore, the conversion is based on robust, the real-aerosol-world reflecting conversion factors. And the method can be used even in the upper troposphere and stratosphere
The classical 6+2 or 3+2 inversion lidar technique is only applicable to the polluted continental or marine boundary layer and pronounced smoke, pollution, or volcanic laers in the free troposphere and stratosphere. But in the majority of free tropospheric and stratospheric observations the aerosol extinction coefficients are too low, too uncertain to apply the inversion technique in a trustworthy way. Therefore, the easy-to-apply POLIPHON method was developed in order to provide always aerosol (CCN, INP) information at cloud level, even at cirrus level close to the tropopause. This is big advantage to all multiwavelength inversion techniques and that should not be forgotten in discussions of all these lidar retrieval methods.
Final point: What about Mie scattering applications in cases of lidar observations over northern Africa, western to eastern Asia, and western North America, where non-spherical dust particles significantly influence the measured backscatter and extinction profiles? Especially backscattering is a rather sensitive function of the shape characteristics as the paper of Haarig et al. (ACP 2022, 3+3 dust observations over Leipzig and comparison to AERONET inversion products) clearly showed. In contrast to the inversion methods, the POLIPHON approach has no problems with observations over dusty areas. In contrast, when using multiwavelength inversion methods and applying the polarization lidar techniques to separate dust and non-dust aerosol fractions, you will need an ML approach based on Mie and non-Mie scattering data bases. Any idea how to realize this?
Sorry for my critical remarks. I just want to foster a more realistic view on the entire inversion problematic.