the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 26 Apr 2024
Improving solution availability and temporal consistency of an optimal estimation physical retrieval for ground-based thermodynamic boundary layer profiling
Abstract. Thermodynamic profiles in the atmospheric boundary layer can be retrieved from ground-based passive remote sensing instruments like infrared spectrometers and microwave radiometers with optimal estimation physical retrievals. With a high temporal resolution on the order of minutes, these thermodynamic profiles are a powerful tool to study the evolution of the boundary layer and to evaluate numerical models. In this study, we present three recent modifications to the Tropospheric Remotely Observed Profiling via Optimal Estimation (TROPoe) retrieval framework, which improve the availability of valid solutions for different atmospheric conditions and increase the temporal consistency of the retrieved profiles. The characterization of the uncertainty of the input and the choice of spectral infrared bands is crucial for retrieval performance and we present methods to enhance the availability of valid solutions retrieved from infrared spectrometers by preventing overfitting and by adding information from an additional spectral band in high moisture environments. Since each profile is retrieved independently from the previous one, the time series of the thermodynamic variables contain random uncorrelated noise, which may hinder the study of diurnal cycles and temporal tendencies. By including a previous retrieved profile as input to the retrieval, we increase the temporal consistency between subsequent profiles without suppressing real mesoscale atmospheric variability. We demonstrate that these modifications work well at mid-latitudes, polar, and tropical sites and for retrievals based on infrared spectrometers and microwave radiometers measurements.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(5290 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(5290 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-714', Anonymous Referee #1, 22 May 2024
The manuscript presents improvements in a retrieval algorithm for ground-based thermodynamic profiles in the boundary layer. It is well structured and clear in the goals of the study as well as in the presentation of the results. I found the explanations and the illustration of the methodology well referenced and convincingly justified. I recommend the publication of the manuscript with just few minor/technical corrections.
Minor comments
- L61-63. I find this part a bit confusing as you just mentioned the need to inflate the noise and you apply a noise-reduction technique. Maybe to make the sentence clearer, I would reformulate it in this way: “The usage of the radiance uncertainty before noise filtering for the error covariance matrix together with the noise-filtered radiance in the measurement vector is intended to compensate for the missing forward model uncertainty”.
- L181-182. Can you clarify the usage of cloudy-contaminated data for the analysis? In particular, in the last paragraph of Sect. 2 it is not clear to me whether you use only cloud-free profiles for the IRS analysis or, as you said before in the manuscript, you keep cloud-free data only for the radiosonde comparison.
- Two questions about the WVBAND experiment. Is the information about near-surface water vapor coming from Ymet? I understand that you use the information from the additional band according to the WV content in a linear fashion, but could this usage introduce an overall bias between dry air profiles and high-humidity profiles? Is this additional band used for all retrievals in Fig. 3 panel (c)? I notice that almost all values are changing in panel (c) with respect to panel (b), and I assume that the only change between the two is the additional band.
- Regarding the TROPEIN experiment: is the usage of the additional information at a previous step in the measurement vector equivalent to using as a-priori information the retrieved profile at a previous time step? Or would this make the retrieval too tight to the previous state?
Technical comments
L7: is crucial → are crucial
L6-10: I suggest to move the sentence “The characterization of the uncertainty … for retrieval performance” right before “Since each profile…” and start here a new sentence “We present methods…”
L14: spectrometers, radiometers → spectrometer, radiometer
L16-17: I would reformulate as: “Observations of the continuous temporal evolution and the diurnal cycle of thermodynamic profiles are essential for the analysis of physical processes….”
L30: Shall you also spell AERIoe out?
L46: I would replace “this process is iteratively repeated” with “the state vector is modified in an iterative process.”
L76: spectral band from → spectral band at
L95: are analyzed → is analyzed
L127: I would say “once daily only during intense observation periods”
L127: numbers → number
L133: I think the detail about the usage of narrow and wide FOVs is possibly too technical, if you don’t explain it further, I think it is better to just say that the detail of the usage of water vapor profiles is described in the papers you mention.
L164. Isn’t the 1-σ uncertainty the square root of the diagonal elements of the matrix Sop?
Fig.7 and 8: Since you first describe Fig.8 and then discuss Fig.7 (except for the reminder at L257) I would invert the order of the two figures.
End of Sect. 3, I would add a sentence informing that the results of the correlation analysis are presented in the next section.
L355: on the average → on average
Caption of Fig.11: I find the term “probability” confusing; would it be appropriate to say “distribution of”?
L383: linesin → lines in
L384: lower → less relevant
L386: Like for → In the same way as for
L387: Please introduce again Fig.11 here, for example: “As reported in Fig. 11 bottom row, …”
L424: contain → have
L424: an additional spectral band → the additional spectral band
L425: add “ratio” to water vapor mixing
L427: I would delete the comma after “water vapor band”
L430: “10-min profile” → “10 minutes a profile” …
In the caption of Fig. A1, replace (Fig. A1)” with “(panel a)”
Citation: https://doi.org/10.5194/egusphere-2024-714-RC1 -
AC1: 'Reply on RC1', Bianca Adler, 17 Jul 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-714/egusphere-2024-714-AC1-supplement.pdf
-
RC2: 'Comment on egusphere-2024-714', Anonymous Referee #2, 27 Jun 2024
This paper outlines three key improvements to the TROPoe retrieval algorithm, a software package that is seeing increasing use in both operational and research milieus. These three improvements, namely the addition of a water vapor band to the retrieval, the integration of the t–1 retrieval to help improve temporal consistency and reduce striping, and the impact of radiance noise inflation, are all discussed. Overall, this is a well-written and compelling manuscript that fits with the scope of AMT and is suitable for publication after a number of small issues are addressed. These are mostly issues associated with explanations and justifications.
The most significant issue I see is in the conversation about inflating the radiance noise to account for the fact that the model error is not expressly addressed. This invokes a somewhat lengthly list of questions, but in its current form the manuscript could do more to justify why this approach is proper and valid. Are uncertainties really fungible like that? Can one inflate one set of uncertainties and assume that it encompasses a different set of uncertainties that exist for an entirely different set of reasons? What are the limitations on including model uncertainties in the retrieval (i.e. just how expensive is it to do it explicitly, and by what factor is the outlined approach better)? Has there ever been an attempt to treat the TROPoe model errors explicitly, and if so, how do those results compare to the noise inflation approach? What is the purpose of reducing the noise with the PCA filter if one is just going to inflate it right back up again? Why is the number of converged retrievals the appropriate measure to determine if the proper inflation factor has been reached?
Some other smaller issues:
In many cases, the IRSes and MWRs used in this study are not at the same location, but instead are located within the same general climate regime. Does that have any impact, i.e.can we compare the moisture variability for the tropical IRS to that of the MWR? My guess is that it's fine, but it probably should be discussed.
Line 119: is it "fore optics," "fore-optics," or "foreoptics?" I've seen all three, but I think I've seen the last one the most.
Line 129, Table 1: The geographic column would benefit by also adding some place names ("Oklahoma USA," "Brazil," "Greenland," etc.) Also, the parentheses in the Number of Radiosondes column are mismatched.
Line 158: Specify that the 55 levels in thermodynamic profile retrievals are for TROPoe; as it is written, it sounds like it's the case for all thermodynamic retrievals regardless of instrument.
Lines 216-226: Adding the WV band may help the number of converged retrievals, but is there an impact on their accuracy? Moreover, is there a discernable impact on the performance of the T retrievals in addition to the WV retrievals?
Line 251: The temporal consistency between the atmosphere is going to vary based on the diurnal cycle. Are there plans to vary the noise inflation uncertainty of the previous retrieval based on time of day?
Lines 255-259: how were the specific values for N decided?
Line 276: If the processing is typically executed independently for each day, then when looking at continuous time series that span the 0000 UTC hour, there will be an artifact of increased variability for some discernible time period every day. Can the algorithm be modified to take into account retrievals from the previous day?
Line 368: fewer, not less
Line 470: Remove the comma between noise and decreases.
Citation: https://doi.org/10.5194/egusphere-2024-714-RC2 -
AC2: 'Reply on RC2', Bianca Adler, 17 Jul 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-714/egusphere-2024-714-AC2-supplement.pdf
-
AC2: 'Reply on RC2', Bianca Adler, 17 Jul 2024
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-714', Anonymous Referee #1, 22 May 2024
The manuscript presents improvements in a retrieval algorithm for ground-based thermodynamic profiles in the boundary layer. It is well structured and clear in the goals of the study as well as in the presentation of the results. I found the explanations and the illustration of the methodology well referenced and convincingly justified. I recommend the publication of the manuscript with just few minor/technical corrections.
Minor comments
- L61-63. I find this part a bit confusing as you just mentioned the need to inflate the noise and you apply a noise-reduction technique. Maybe to make the sentence clearer, I would reformulate it in this way: “The usage of the radiance uncertainty before noise filtering for the error covariance matrix together with the noise-filtered radiance in the measurement vector is intended to compensate for the missing forward model uncertainty”.
- L181-182. Can you clarify the usage of cloudy-contaminated data for the analysis? In particular, in the last paragraph of Sect. 2 it is not clear to me whether you use only cloud-free profiles for the IRS analysis or, as you said before in the manuscript, you keep cloud-free data only for the radiosonde comparison.
- Two questions about the WVBAND experiment. Is the information about near-surface water vapor coming from Ymet? I understand that you use the information from the additional band according to the WV content in a linear fashion, but could this usage introduce an overall bias between dry air profiles and high-humidity profiles? Is this additional band used for all retrievals in Fig. 3 panel (c)? I notice that almost all values are changing in panel (c) with respect to panel (b), and I assume that the only change between the two is the additional band.
- Regarding the TROPEIN experiment: is the usage of the additional information at a previous step in the measurement vector equivalent to using as a-priori information the retrieved profile at a previous time step? Or would this make the retrieval too tight to the previous state?
Technical comments
L7: is crucial → are crucial
L6-10: I suggest to move the sentence “The characterization of the uncertainty … for retrieval performance” right before “Since each profile…” and start here a new sentence “We present methods…”
L14: spectrometers, radiometers → spectrometer, radiometer
L16-17: I would reformulate as: “Observations of the continuous temporal evolution and the diurnal cycle of thermodynamic profiles are essential for the analysis of physical processes….”
L30: Shall you also spell AERIoe out?
L46: I would replace “this process is iteratively repeated” with “the state vector is modified in an iterative process.”
L76: spectral band from → spectral band at
L95: are analyzed → is analyzed
L127: I would say “once daily only during intense observation periods”
L127: numbers → number
L133: I think the detail about the usage of narrow and wide FOVs is possibly too technical, if you don’t explain it further, I think it is better to just say that the detail of the usage of water vapor profiles is described in the papers you mention.
L164. Isn’t the 1-σ uncertainty the square root of the diagonal elements of the matrix Sop?
Fig.7 and 8: Since you first describe Fig.8 and then discuss Fig.7 (except for the reminder at L257) I would invert the order of the two figures.
End of Sect. 3, I would add a sentence informing that the results of the correlation analysis are presented in the next section.
L355: on the average → on average
Caption of Fig.11: I find the term “probability” confusing; would it be appropriate to say “distribution of”?
L383: linesin → lines in
L384: lower → less relevant
L386: Like for → In the same way as for
L387: Please introduce again Fig.11 here, for example: “As reported in Fig. 11 bottom row, …”
L424: contain → have
L424: an additional spectral band → the additional spectral band
L425: add “ratio” to water vapor mixing
L427: I would delete the comma after “water vapor band”
L430: “10-min profile” → “10 minutes a profile” …
In the caption of Fig. A1, replace (Fig. A1)” with “(panel a)”
Citation: https://doi.org/10.5194/egusphere-2024-714-RC1 -
AC1: 'Reply on RC1', Bianca Adler, 17 Jul 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-714/egusphere-2024-714-AC1-supplement.pdf
-
RC2: 'Comment on egusphere-2024-714', Anonymous Referee #2, 27 Jun 2024
This paper outlines three key improvements to the TROPoe retrieval algorithm, a software package that is seeing increasing use in both operational and research milieus. These three improvements, namely the addition of a water vapor band to the retrieval, the integration of the t–1 retrieval to help improve temporal consistency and reduce striping, and the impact of radiance noise inflation, are all discussed. Overall, this is a well-written and compelling manuscript that fits with the scope of AMT and is suitable for publication after a number of small issues are addressed. These are mostly issues associated with explanations and justifications.
The most significant issue I see is in the conversation about inflating the radiance noise to account for the fact that the model error is not expressly addressed. This invokes a somewhat lengthly list of questions, but in its current form the manuscript could do more to justify why this approach is proper and valid. Are uncertainties really fungible like that? Can one inflate one set of uncertainties and assume that it encompasses a different set of uncertainties that exist for an entirely different set of reasons? What are the limitations on including model uncertainties in the retrieval (i.e. just how expensive is it to do it explicitly, and by what factor is the outlined approach better)? Has there ever been an attempt to treat the TROPoe model errors explicitly, and if so, how do those results compare to the noise inflation approach? What is the purpose of reducing the noise with the PCA filter if one is just going to inflate it right back up again? Why is the number of converged retrievals the appropriate measure to determine if the proper inflation factor has been reached?
Some other smaller issues:
In many cases, the IRSes and MWRs used in this study are not at the same location, but instead are located within the same general climate regime. Does that have any impact, i.e.can we compare the moisture variability for the tropical IRS to that of the MWR? My guess is that it's fine, but it probably should be discussed.
Line 119: is it "fore optics," "fore-optics," or "foreoptics?" I've seen all three, but I think I've seen the last one the most.
Line 129, Table 1: The geographic column would benefit by also adding some place names ("Oklahoma USA," "Brazil," "Greenland," etc.) Also, the parentheses in the Number of Radiosondes column are mismatched.
Line 158: Specify that the 55 levels in thermodynamic profile retrievals are for TROPoe; as it is written, it sounds like it's the case for all thermodynamic retrievals regardless of instrument.
Lines 216-226: Adding the WV band may help the number of converged retrievals, but is there an impact on their accuracy? Moreover, is there a discernable impact on the performance of the T retrievals in addition to the WV retrievals?
Line 251: The temporal consistency between the atmosphere is going to vary based on the diurnal cycle. Are there plans to vary the noise inflation uncertainty of the previous retrieval based on time of day?
Lines 255-259: how were the specific values for N decided?
Line 276: If the processing is typically executed independently for each day, then when looking at continuous time series that span the 0000 UTC hour, there will be an artifact of increased variability for some discernible time period every day. Can the algorithm be modified to take into account retrievals from the previous day?
Line 368: fewer, not less
Line 470: Remove the comma between noise and decreases.
Citation: https://doi.org/10.5194/egusphere-2024-714-RC2 -
AC2: 'Reply on RC2', Bianca Adler, 17 Jul 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-714/egusphere-2024-714-AC2-supplement.pdf
-
AC2: 'Reply on RC2', Bianca Adler, 17 Jul 2024
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 291 | 81 | 27 | 399 | 12 | 17 |
- HTML: 291
- PDF: 81
- XML: 27
- Total: 399
- BibTeX: 12
- EndNote: 17
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Cited
David D. Turner
Laura Bianco
Irina V. Djalalova
Timothy Myers
James M. Wilczak
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(5290 KB) - Metadata XML