Aerosol extinction and backscatter Optimal Estimation retrieval for High Spectral Resolution Lidar

Burton, Sharon P.; Hair, Johnathan W.; Hostetler, Chris A.; Fenn, Marta A.; Smith, John A.; Ferrare, Richard A.

doi:https://doi.org/10.5194/egusphere-2025-2915

Preprints

https://doi.org/10.5194/egusphere-2025-2915

Preprints

11 Jul 2025

| 11 Jul 2025

Aerosol extinction and backscatter Optimal Estimation retrieval for High Spectral Resolution Lidar

Sharon P. Burton, Johnathan W. Hair, Chris A. Hostetler, Marta A. Fenn, John A. Smith, and Richard A. Ferrare

Abstract. High Spectral Resolution Lidars (HSRLs) have been successfully deployed from a variety of platforms: ground based, airborne, and now satellite. These lidars are uniquely valuable for characterizing atmospheric aerosol and clouds, benefiting from the ability to characterize vertical variability in more detail than any passive instruments, and, compared to elastic backscatter lidars, provide additional channels of measurements that permit the direct retrieval of particulate extinction. Although analytic solutions exist for deriving particulate backscatter, extinction, and linear depolarization ratio, in the case of extinction, the analytic technique greatly magnifies measurement noise. Low signal-to-noise measurements stress the traditional inversion methods. Accordingly, algorithms for the retrieval of HSRL backscatter and extinction are re-examined and optimized to reduce the noise propagation. Here we explore an Optimal Estimation methodology and compare it with an implementation of the direct differentiation method like that historically used for the processing of airborne HSRL data from NASA Langley Research Center.

Received: 19 Jun 2025 – Discussion started: 11 Jul 2025

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors. Views expressed in the text are those of the authors and do not necessarily reflect the views of the publisher.

Download & links

Sharon P. Burton, Johnathan W. Hair, Chris A. Hostetler, Marta A. Fenn, John A. Smith, and Richard A. Ferrare

Status: closed

RC1:
'Comment on egusphere-2025-2915', Anonymous Referee #2, 28 Jul 2025

I should start by saying, that this is an excellent and deep research. Calculating the the aerosol extinction coefficient is a challenging task, particularly in the presence of noise. The authors convincingly demonstrate that the Optimal Estimation (OE) technique reduces uncertainty in the calculations compared to traditional analytical methods. Additionally, the approach enables the determination of the optimal height resolution for different altitude ranges. Another key strength is its ability to account for both random and systematic uncertainties, including calibration errors.
Overall, this is a high-quality scientific manuscript that is certainly publishable in its current form. However, while certain aspects may seem self-evident to the authors, readers less familiar with OE may benefit from additional explanations.

Technical comments
Eq.4. Symbol “C” is the same as in Eq.1?

p.7 ln.18 “If the residual is approximately one or less, then the solution agrees well…”. Would be good to explain or provide a reference.

p.9 ln 19. “the prior profile of backscatter is taken to be zero”. Probably needs explanation, why it is zero. Just wonder, if a prior profile can be calculated from standard analytical approach.

Fig.5. What are the units? Are measurements normalized?

Fig.11. I have difficulty to understand this figure. Why effective resolution in OE method increases in non-monotonic way? For grid of 50 m the uncertainty is the highest, but effective resolution is also high (700 m). Would be good to provide more explanations.

Fig.14f. At altitude of ~0.5 km uncertainty of OE is higher than for analytical solution. Can it be explained?

Citation: https://doi.org/10.5194/egusphere-2025-2915-RC1
- AC2: 'Reply on RC1', Sharon Burton, 06 Sep 2025
  
  Thank you to the reviewers for your helpful suggestions. The responses have been attached as a pdf, for easier formatting.
  
  Citation: https://doi.org/10.5194/egusphere-2025-2915-AC2
RC2:
'Comment on egusphere-2025-2915', Anonymous Referee #1, 30 Jul 2025
The article describes an algorithm for an improved extinction retrieval from high spectral resolution lidar (HSRL) measurements. An analytical solution exists for the extinction coefficient derived from HSRL measurements, which often suffers from low signal to noise ratios. The presented optimal estimation solution has the benefit of less noisy extinction coefficient and lidar ratio profiles, which is desirable for aerosol research with lidar. Similar approaches were already reported for Raman lidar measurements and spaceborne HSRL observations. Here, the approach is developed for the airborne HSRL of NASA Langley Research Center. Another highlight is the characterisation of measurement uncertainties. The article is clearly written and describes the algorithm and applies it to simulated and real measurement data. The topic fits in the scope of AMT and should be published after minor revisions which mostly concern the figures.
Minor comments:
While the article focusses on the application to airborne HSRL systems, the possibility of the application to ground-based HSR lidars should be considered. New ground-based HSRL systems were developed, e.g., Jin et al., 2020.

Jin, Y.; Nishizawa, T.; Sugimoto, N.; Takakura, S.; Aoki, M.; Ishii, S.; Yamazaki, A.; Kudo, R.; Yumimoto, K.; Sato, K. & Okamoto, H.: Demonstration of aerosol profile measurement with a dual-wavelength high-spectral-resolution lidar using a scanning interferometer, Appl. Opt., Optica Publishing Group, 2022, 61, 3523-3532

P4L20: Here, you call the lidar system HSRL-2, later just HSRL2. It is up to you to decide how to name your lidar system.

Eq 11 is not really an equation (no =). Furthermore, the journal standards expect a vector to be in bold italic.

In nearly all figures, the units of the optical properties are missing.

Furthermore, the figures are often quite small. Sometimes, the minor ticks are not resolved well.

Fig 2 + 12: The perpendicular signal (in green) is multiplied by a factor of 10, not the particle signal. The label inside the figure is wrong.

Fig 2: There are no pink circles as mentioned in the caption.

You use the terms “particle/particulate” and “aerosol” synonymously in the description of the optical properties and signals, e.g., particulate depolarization ratio and aerosol depolarization ratio or particle(-dominated) signal or aerosol signal. It is ok, if you focus on aerosol and exclude clouds. However, it would be nice to harmonize the terms throughout the manuscript and especially the axis labels in the figures. Sometimes, the particle dominated channel is referred to as “para” – parallel in the axis labels (e.g., Fig 5+18).

The green lines in all your figures look almost blue to me. Maybe you find a different type of green to be clear.

P12L16-17 Why the systematic uncertainties are not included in the analytic retrieval?

P14L13-14: Probably, this is the reason why the analytic retrieval has the lowest residual in the topmost kilometer for all three signals.

P14L17 Fig 1 does not contain uncertainty estimates. Probably, you want to refer to a different figure here.

Fig 6 + 15: I would add a, b, c … to the subplots.

Please estimate how important are Fig 7+8 and Fig 16 + 17. Maybe a description in the text is sufficient. It is up to you to decide.

May I suggest, furthermore, to combine Fig 10 and 11 as Fig 10a and 10b?

Fig 18: The exponents 10^x are hard to read.

Please add a section about the code availability.

Overall, it is a well-written manuscript which presents an important improvement in the analysis of HSRL extinction data. The method provides great benefit for the research community and should be published after some minor revisions. Well done.
Citation: https://doi.org/10.5194/egusphere-2025-2915-RC2
- AC1: 'Reply on RC2', Sharon Burton, 06 Sep 2025
  
  Thank you to the reviewers for your helpful suggestions. The responses have been attached as a pdf, for easier formatting.
  
  Citation: https://doi.org/10.5194/egusphere-2025-2915-AC1
RC3:
'Comment on egusphere-2025-2915', Anonymous Referee #3, 31 Jul 2025
Review of "Aerosol extinction and backscatter optimal estimation retrieval for high spectral resolution lidar" by S. P. Burton, J. W. Hair, C. A. Hostetler, M. A. Fenn, J. A. Smith, and R. A. Ferrare, proposed for publication in Atmospheric Measurement Techniques.

In this article, the authors present a method based on optimal estimation to retrieve lidar-based optical parameters (particulate backscatter, extinction, lidar ratio, depolarization ratio) in aerosols based on HSRL measurements in 3 channels (cross-polarized, and co-polarized Mie-dominated and Rayleigh-dominated). They compare results from that approach to those obtained using the more widely used analytical approach. They consider a theoretical case study, and actual measurements from the NASA Langley airborne HSRL instrument in a scene that mixes different aerosol layers. They discuss the advantages of the optimal estimation approach for each retrieval parameter, and introduce some of its peculiarities, like the effective vertical resolution.

The subject matter is new, interesting and valuable for lidar experts, especially now that HSRL measurements from space are widely available. The writing is clear and effective. The methodology is well-supported by appropriate references. I appreciate that the authors took the time to remind readers of the basics, and enlightening comments can be found throughout the manuscript. Figures are well-designed, most convey a clear and useful message with evidence that moves the discussion forward. The results show that the method proposed by the authors brings significant improvements to retrievals of aerosol extinction compared to the analytical approach. Even though the paper could very well be published as-is, I have minor comments that mainly hope to make the paper slightly more approachable to non-specialists.

Minor Comments

Figs. 1, 2, 3, 6, 7, 8, 12, 14, 15, 16, 17 : please add units for parameters when relevant (backscatter, lidar ratio, extinction, altitude)

p. 6, L.15: "the backscatter and depolarization are each found from the ratio of channels" - how is the backscatter obtained from the ratio of channels? I could not find how this aligns with eq 4 in Hair 2008 or any equation in Burton 2018. As I understand it the backscatter is directly proportional either to the Pm (for molecular backscatter) or Pp (for particulate backscatter) channels, no ratio here.

Section 3 is huge (17 pages out of 28, not counting the references). Splitting 3.1 and 3.2 into their own sections could help make things more balanced.

The authors introduce in Section 3.1.3 the "effective resolution", which is different from the grid resolution and is given by the inverse of the degree of freedom. From the text I understand that having a different effective resolution for each data point along the vertical profile is a consequence of the optimal estimation approach considering entire profiles of all parameters at once (compared to the analytical solution which considers each altitude point independently). I also understand the effective resolution gets coarser where the signal is weaker and the optimal estimation gives precedence to the prior compared to the measurement. I'm not sure, however, of how to interpret a profile of effective vertical resolution as in Figures 9 or 19. In Figure 19, the effective resolution reaches > 1 km near 500m ASL, but is much finer (< 500 m) 165m above or 165m below. Is the effective resolution only to be interpreted qualitatively as an indicator of the relative importances of the signal and the prior at that particular height, or do the values themselves (500m, 1 km) mean something? If so, could you expand a bit of how to interpret them? If not, would it make sense to divide the minimum possible value (165m) by the effective resolution and provide the result as a unitless qualitative indicator of the importance of the signal for the retrieval at each height?

Fig. 12: I'm not sure what is the point of using diamonds, squares and circles -- at their size the symbols are almost undistinguishable anyways (even in the legend).

Even though the figures are well-designed and extremely clear, 19 figures is a lot, and many have a lot of subplots. Maybe some of the figures could be omitted. I'll admit that the point of the correlation plots (6, 7, 8, 15, 16, 17) was lost on me. Maybe the authors could sum up textually whatever conclusions they drew from these figures (eg about where the uncertainties are mainly systematic) and move the figures to an appendix?

Have the authors planned to make their data analysis package (that does the optimal estimation based on lidar measurements as input) available online, for instance as a python package or something equivalent? Since the improvements are so significant compared to the more widely-used analytical approach, doing so would clearly benefit the whole lidar-based community.
Citation: https://doi.org/10.5194/egusphere-2025-2915-RC3
- AC3: 'Reply on RC3', Sharon Burton, 06 Sep 2025
  
  Thank you to the reviewers for your helpful suggestions. The responses have been attached as a pdf, for easier formatting.
  
  Citation: https://doi.org/10.5194/egusphere-2025-2915-AC3

Status: closed

RC1:
'Comment on egusphere-2025-2915', Anonymous Referee #2, 28 Jul 2025

I should start by saying, that this is an excellent and deep research. Calculating the the aerosol extinction coefficient is a challenging task, particularly in the presence of noise. The authors convincingly demonstrate that the Optimal Estimation (OE) technique reduces uncertainty in the calculations compared to traditional analytical methods. Additionally, the approach enables the determination of the optimal height resolution for different altitude ranges. Another key strength is its ability to account for both random and systematic uncertainties, including calibration errors.
Overall, this is a high-quality scientific manuscript that is certainly publishable in its current form. However, while certain aspects may seem self-evident to the authors, readers less familiar with OE may benefit from additional explanations.

Technical comments
Eq.4. Symbol “C” is the same as in Eq.1?

p.7 ln.18 “If the residual is approximately one or less, then the solution agrees well…”. Would be good to explain or provide a reference.

p.9 ln 19. “the prior profile of backscatter is taken to be zero”. Probably needs explanation, why it is zero. Just wonder, if a prior profile can be calculated from standard analytical approach.

Fig.5. What are the units? Are measurements normalized?

Fig.11. I have difficulty to understand this figure. Why effective resolution in OE method increases in non-monotonic way? For grid of 50 m the uncertainty is the highest, but effective resolution is also high (700 m). Would be good to provide more explanations.

Fig.14f. At altitude of ~0.5 km uncertainty of OE is higher than for analytical solution. Can it be explained?

Citation: https://doi.org/10.5194/egusphere-2025-2915-RC1
- AC2: 'Reply on RC1', Sharon Burton, 06 Sep 2025
  
  Thank you to the reviewers for your helpful suggestions. The responses have been attached as a pdf, for easier formatting.
  
  Citation: https://doi.org/10.5194/egusphere-2025-2915-AC2
RC2:
'Comment on egusphere-2025-2915', Anonymous Referee #1, 30 Jul 2025
The article describes an algorithm for an improved extinction retrieval from high spectral resolution lidar (HSRL) measurements. An analytical solution exists for the extinction coefficient derived from HSRL measurements, which often suffers from low signal to noise ratios. The presented optimal estimation solution has the benefit of less noisy extinction coefficient and lidar ratio profiles, which is desirable for aerosol research with lidar. Similar approaches were already reported for Raman lidar measurements and spaceborne HSRL observations. Here, the approach is developed for the airborne HSRL of NASA Langley Research Center. Another highlight is the characterisation of measurement uncertainties. The article is clearly written and describes the algorithm and applies it to simulated and real measurement data. The topic fits in the scope of AMT and should be published after minor revisions which mostly concern the figures.
Minor comments:
While the article focusses on the application to airborne HSRL systems, the possibility of the application to ground-based HSR lidars should be considered. New ground-based HSRL systems were developed, e.g., Jin et al., 2020.

Jin, Y.; Nishizawa, T.; Sugimoto, N.; Takakura, S.; Aoki, M.; Ishii, S.; Yamazaki, A.; Kudo, R.; Yumimoto, K.; Sato, K. & Okamoto, H.: Demonstration of aerosol profile measurement with a dual-wavelength high-spectral-resolution lidar using a scanning interferometer, Appl. Opt., Optica Publishing Group, 2022, 61, 3523-3532

P4L20: Here, you call the lidar system HSRL-2, later just HSRL2. It is up to you to decide how to name your lidar system.

Eq 11 is not really an equation (no =). Furthermore, the journal standards expect a vector to be in bold italic.

In nearly all figures, the units of the optical properties are missing.

Furthermore, the figures are often quite small. Sometimes, the minor ticks are not resolved well.

Fig 2 + 12: The perpendicular signal (in green) is multiplied by a factor of 10, not the particle signal. The label inside the figure is wrong.

Fig 2: There are no pink circles as mentioned in the caption.

You use the terms “particle/particulate” and “aerosol” synonymously in the description of the optical properties and signals, e.g., particulate depolarization ratio and aerosol depolarization ratio or particle(-dominated) signal or aerosol signal. It is ok, if you focus on aerosol and exclude clouds. However, it would be nice to harmonize the terms throughout the manuscript and especially the axis labels in the figures. Sometimes, the particle dominated channel is referred to as “para” – parallel in the axis labels (e.g., Fig 5+18).

The green lines in all your figures look almost blue to me. Maybe you find a different type of green to be clear.

P12L16-17 Why the systematic uncertainties are not included in the analytic retrieval?

P14L13-14: Probably, this is the reason why the analytic retrieval has the lowest residual in the topmost kilometer for all three signals.

P14L17 Fig 1 does not contain uncertainty estimates. Probably, you want to refer to a different figure here.

Fig 6 + 15: I would add a, b, c … to the subplots.

Please estimate how important are Fig 7+8 and Fig 16 + 17. Maybe a description in the text is sufficient. It is up to you to decide.

May I suggest, furthermore, to combine Fig 10 and 11 as Fig 10a and 10b?

Fig 18: The exponents 10^x are hard to read.

Please add a section about the code availability.

Overall, it is a well-written manuscript which presents an important improvement in the analysis of HSRL extinction data. The method provides great benefit for the research community and should be published after some minor revisions. Well done.
Citation: https://doi.org/10.5194/egusphere-2025-2915-RC2
- AC1: 'Reply on RC2', Sharon Burton, 06 Sep 2025
  
  Thank you to the reviewers for your helpful suggestions. The responses have been attached as a pdf, for easier formatting.
  
  Citation: https://doi.org/10.5194/egusphere-2025-2915-AC1
RC3:
'Comment on egusphere-2025-2915', Anonymous Referee #3, 31 Jul 2025
Review of "Aerosol extinction and backscatter optimal estimation retrieval for high spectral resolution lidar" by S. P. Burton, J. W. Hair, C. A. Hostetler, M. A. Fenn, J. A. Smith, and R. A. Ferrare, proposed for publication in Atmospheric Measurement Techniques.

In this article, the authors present a method based on optimal estimation to retrieve lidar-based optical parameters (particulate backscatter, extinction, lidar ratio, depolarization ratio) in aerosols based on HSRL measurements in 3 channels (cross-polarized, and co-polarized Mie-dominated and Rayleigh-dominated). They compare results from that approach to those obtained using the more widely used analytical approach. They consider a theoretical case study, and actual measurements from the NASA Langley airborne HSRL instrument in a scene that mixes different aerosol layers. They discuss the advantages of the optimal estimation approach for each retrieval parameter, and introduce some of its peculiarities, like the effective vertical resolution.

The subject matter is new, interesting and valuable for lidar experts, especially now that HSRL measurements from space are widely available. The writing is clear and effective. The methodology is well-supported by appropriate references. I appreciate that the authors took the time to remind readers of the basics, and enlightening comments can be found throughout the manuscript. Figures are well-designed, most convey a clear and useful message with evidence that moves the discussion forward. The results show that the method proposed by the authors brings significant improvements to retrievals of aerosol extinction compared to the analytical approach. Even though the paper could very well be published as-is, I have minor comments that mainly hope to make the paper slightly more approachable to non-specialists.

Minor Comments

Figs. 1, 2, 3, 6, 7, 8, 12, 14, 15, 16, 17 : please add units for parameters when relevant (backscatter, lidar ratio, extinction, altitude)

p. 6, L.15: "the backscatter and depolarization are each found from the ratio of channels" - how is the backscatter obtained from the ratio of channels? I could not find how this aligns with eq 4 in Hair 2008 or any equation in Burton 2018. As I understand it the backscatter is directly proportional either to the Pm (for molecular backscatter) or Pp (for particulate backscatter) channels, no ratio here.

Section 3 is huge (17 pages out of 28, not counting the references). Splitting 3.1 and 3.2 into their own sections could help make things more balanced.

The authors introduce in Section 3.1.3 the "effective resolution", which is different from the grid resolution and is given by the inverse of the degree of freedom. From the text I understand that having a different effective resolution for each data point along the vertical profile is a consequence of the optimal estimation approach considering entire profiles of all parameters at once (compared to the analytical solution which considers each altitude point independently). I also understand the effective resolution gets coarser where the signal is weaker and the optimal estimation gives precedence to the prior compared to the measurement. I'm not sure, however, of how to interpret a profile of effective vertical resolution as in Figures 9 or 19. In Figure 19, the effective resolution reaches > 1 km near 500m ASL, but is much finer (< 500 m) 165m above or 165m below. Is the effective resolution only to be interpreted qualitatively as an indicator of the relative importances of the signal and the prior at that particular height, or do the values themselves (500m, 1 km) mean something? If so, could you expand a bit of how to interpret them? If not, would it make sense to divide the minimum possible value (165m) by the effective resolution and provide the result as a unitless qualitative indicator of the importance of the signal for the retrieval at each height?

Fig. 12: I'm not sure what is the point of using diamonds, squares and circles -- at their size the symbols are almost undistinguishable anyways (even in the legend).

Even though the figures are well-designed and extremely clear, 19 figures is a lot, and many have a lot of subplots. Maybe some of the figures could be omitted. I'll admit that the point of the correlation plots (6, 7, 8, 15, 16, 17) was lost on me. Maybe the authors could sum up textually whatever conclusions they drew from these figures (eg about where the uncertainties are mainly systematic) and move the figures to an appendix?

Have the authors planned to make their data analysis package (that does the optimal estimation based on lidar measurements as input) available online, for instance as a python package or something equivalent? Since the improvements are so significant compared to the more widely-used analytical approach, doing so would clearly benefit the whole lidar-based community.
Citation: https://doi.org/10.5194/egusphere-2025-2915-RC3
- AC3: 'Reply on RC3', Sharon Burton, 06 Sep 2025
  
  Thank you to the reviewers for your helpful suggestions. The responses have been attached as a pdf, for easier formatting.
  
  Citation: https://doi.org/10.5194/egusphere-2025-2915-AC3

Sharon P. Burton, Johnathan W. Hair, Chris A. Hostetler, Marta A. Fenn, John A. Smith, and Richard A. Ferrare

Viewed

Total article views: 376 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
318	40	18	376	10	13

HTML: 318
PDF: 40
XML: 18
Total: 376
BibTeX: 10
EndNote: 13

Views and downloads (calculated since 11 Jul 2025)

Month	HTML	PDF	XML	Total
Jul 2025	108	20	10	138
Aug 2025	108	12	5	125
Sep 2025	102	8	3	113

Cumulative views and downloads (calculated since 11 Jul 2025)

Month	HTML	PDF	XML	Total
Jul 2025	108	20	10	138
Aug 2025	108	12	5	125
Sep 2025	102	8	3	113

Viewed (geographical distribution)

Total article views: 375 (including HTML, PDF, and XML) Thereof 375 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 11 Sep 2025

Short summary

Advanced lidars can retrieve atmospheric aerosol backscatter, extinction, and linear depolarization ratio. However excessive noise propagation in the traditional inversion method can make extinction unusable when the signal is weak, such as from space. Implementing a weak constraint can reduce the noise propagation while preserving accuracy. Here we also focus on fully characterizing both random and systematic uncertainties and examining the tradeoff between uncertainty and resolution.


Total:	0
HTML:	0
PDF:	0
XML:	0