the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 07 Apr 2026
Optimization of the Fast Layer Transmittance Algorithm in RTTOV v13.1 for Strong Water Vapor Absorption Channels of the FY-3F HIRAS-II Instrument Using LBLRTM v12.11
Abstract. Fast and accurate calculation of atmospheric transmittance is essential for infrared atmospheric remote sensing and satellite data assimilation. However, fast radiative transfer models show significant errors in strong water vapour absorption channels (e.g., near 6.7 μm). An important reason is that the numerical instability encountered during the regression of transmittance coefficients when dealing with lower and middle atmosphere. To address this, this study proposes an optimized scheme to calculate atmospheric transmittance vertical profiles for the RTTOV (Radiative Transfer for TOVS) fast transmittance algorithm. The method introduces a physically motivated transmittance threshold to sub-select training samples and employs cumulative transmittance-based weighting factor within a weighted least squares regression to recalibrate the transmittance coefficients. It aims to optimize the calculation scheme for transmittance coefficients of the Hyperspectral Infrared Atmospheric Sounder-II (HIRAS-II) instrument onboard Fengyun 3F satellite (FY-3F). The method is assessed by calculations on the training profile datasets provided within the RTTOV model framework. By comparing transmittance and brightness temperature calculations at 6.7 microns from this method with those from a line-by-line model and observations from HIRAS-II, the results show that the accuracy of the forward model for the 6.7 μm absorption channel is significantly enhanced by applying a threshold-based noise reduction method. This improvement enhances the stability and reliability of the transmittance calculations for this strong absorption band. Further accuracy enhancements are obtained by incorporating weighting corrections into the calculations of transmittance coefficients. The root mean square error (RMSE) and bias of the observation minus background (OMB) time series for FY-3F HIRAS-II demonstrate that the transmittance coefficient calculation scheme with weighting factor correction improves the forward model accuracy, which is more consistent with RTTOV simulation results. The OMB bias at the 6.7 μm absorption peak channel performs better than that of RTTOV, while the OMB bias on both sides of the 6.7 μm absorption peak channel remains consistent with RTTOV.
- Preprint
(2762 KB) - Metadata XML
- BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on egusphere-2025-5121', Anonymous Referee #1, 11 Apr 2026
-
AC1: 'Reply on RC1', zhang panxiang, 03 Jun 2026
Thank you for your detailed and constructive comments. Below we provide a point-by-point summary of our responses.
1 (Theoretical equivalence vs. practical difference between LSM2 and LSM3):
We fully agree that the weighted least squares method (LSM2) and the ordinary least squares method applied to weighted variables (LSM3) are mathematically equivalent under fixed positive weights and ideal exact arithmetic. Nevertheless, the minor performance difference observed in practical applications originates purely from numerical and procedural factors under finite-precision computations, rather than theoretical discrepancies. Three key factors lead to such numerical deviations. First, this study adopts an iterative reweighting strategy dependent on atmospheric transmittance. The substantial variation of transmittance across multiple orders of magnitude results in a severely ill-conditioned matrix in the normal equation of LSM2, while LSM3 effectively reduces the matrix condition number and improves numerical stability. Second, LSM2 is solved via normal equations, which tends to amplify rounding errors for ill-conditioned matrices. In contrast, LSM3 first implements data transformation and then adopts QR/SVD decomposition solvers with stronger robustness. Third, under extreme conditions with extremely low transmittance, rounding errors in the calculated weights are squared and magnified in LSM2, whereas they are only linearly scaled in LSM3, ultimately causing slight deviations in the iterative convergence results. In summary, the marginal superiority of LSM3 is a pure numerical effect rather than a statistical difference. We have elaborated on this mechanism in detail in the revised manuscript.2 (Capitalization of “When”):
Corrected. On page 5, line 156 and page 6, line 168, “.when” has been changed to “. When”.3 (Derivation of weight factor W ∝ τ):
We have added a physical explanation: transmittance decreases from TOA to surface; lower transmittance implies smaller absorption contribution and lower signal reliability, thus lower weight. After comparing candidate functions, we adopted W_j = |τ_j|. This explanation has been added to the revised manuscript (page 7, lines 184–196).4 (Repetitive paragraphs on page 7):
We have consolidated the two repetitive paragraphs into a single concise paragraph (page 7, lines 215–221 in the revised version). Redundant content has been removed.5 (Threshold 10⁻⁴ for FY-3F HIRAS-II):
We have added quantitative evidence: (1) convolution artifacts from HIRAS-II SRF cause negative transmittance below 10⁻⁴; (2) comparison with LBLRTM shows 10⁻⁴ suppresses errors better than 10⁻⁵; (3) a larger threshold would remove too many atmospheric samples. This threshold is instrument-specific, not a general empirical value. (Revised manuscript page 14, lines 345–352)6 (Figure blurriness and too many profiles):
Figure blurriness was due to resolution loss during file saving. We have upgraded the resolution. Following your suggestion, only three representative profiles (81, 82, 83) are retained in the main figures; the remaining profiles are moved to Appendix A (page 24).7 (Inconsistent figure caption notation):
We have unified all figure captions to use textual descriptions (e.g., “pentagrams”, “circles”) instead of graphic symbols (e.g., “✯”, “○”) throughout the manuscript.All corresponding changes have been clearly marked in the revised manuscript. We appreciate your valuable comments and guidance.
Citation: https://doi.org/10.5194/egusphere-2025-5121-AC1
-
AC1: 'Reply on RC1', zhang panxiang, 03 Jun 2026
-
RC2: 'Comment on egusphere-2025-5121', Anonymous Referee #2, 09 Jun 2026
This manuscript presents an optimization of the fast layer transmittance coefficient calculation for strong water vapor absorption channels of FY-3F HIRAS-II, using LBLRTM v12.11 as a reference and evaluating threshold-based sample selection and weighted least-squares regression schemes. The topic is important for hyperspectral infrared radiance simulation, water-vapor retrieval, and satellite data assimilation, particularly for the 6.7 micrometre band where fast transmittance models can be affected by small-transmittance numerical instability.
The study is technically useful and contains a substantial amount of work, including coefficient training, comparison with line-by-line calculations, and evaluation using HIRAS-II observations. The proposed threshold and weighting approaches appear to improve the forward simulation in the strongest water-vapor channels. I think the manuscript can be strengthened mainly through clearer methodological description, modest sensitivity tests, and a more careful discussion of the observational validation uncertainty.
1. The manuscript would benefit from a clearer explanation of the physical and numerical basis for the 10^-4 transmittance threshold. The current description is plausible, but a compact sensitivity test using a few nearby threshold values would make the choice more convincing. It would be useful to show whether the retained sample numbers, transmittance errors, and brightness-temperature errors remain stable when the threshold is modestly varied.
2. The weighted least-squares schemes LSM2 and LSM3 are promising, but the mathematical description could be made easier to reproduce. Please clarify exactly how the weight factor is defined and whether it is applied to the predictors, the dependent variable, or both before solving the regression. A short algorithmic summary of LSM0-LSM3 would help readers follow the differences among the schemes.
3. Please clarify the RTTOV baseline used for comparison. The title refers to RTTOV v13.1, whereas the validation section mentions RTTOV v13.2. It would be helpful to state the coefficient set, predictors, spectroscopy assumptions, and surface-emissivity treatment used in the baseline comparison so that readers can interpret the magnitude of the reported improvement.
4. The observational comparison with FY-3F HIRAS-II is valuable. Because OMB statistics can also be influenced by clear-sky screening, ERA5 profile errors, surface emissivity, calibration uncertainty, and residual cloud contamination, I suggest adding a short uncertainty discussion. This would make the interpretation of the approximately 0.1 K improvement near the absorption peak more balanced.
5. The clear-sky selection method based on collocated MERSI cloud mask products is a sensible choice. Please provide a little more information on the number of retained HIRAS-II observations, land-ocean distribution, and whether the main conclusions are sensitive to the strict requirement that all MERSI pixels within the HIRAS-II FOV be clear.
6. AI-based approaches are increasingly being explored in fast radiative-transfer and gas-absorption calculations (e.g., https://doi.org/10.1016/j.jqsrt.2026.109920). It would be interesting if the authors could briefly discuss whether the threshold and weighting strategy proposed here might be combined with AI-based approaches in future work to further improve computational efficiency, robustness, or adaptability for strong-absorption-channel simulations.
7. Please correct grammatical and typographical issues throughout. Examples include 'poorly profiles', repeated sentences around the weighting function, and missing spaces before some section headings.
8. Please ensure consistent terminology for HIRAS, HIRAS-II, FY-3E, FY-3F, RTTOV v13.1, and RTTOV v13.2.
9. The rows in Table 1 are not visually well separated, which makes it somewhat difficult to distinguish the different experimental schemes at a glance. The authors may consider improving the table formatting, for example by adding clearer horizontal rules, alternating row shading, or slightly more spacing between rows.
Citation: https://doi.org/10.5194/egusphere-2025-5121-RC2
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 265 | 109 | 26 | 400 | 15 | 23 |
- HTML: 265
- PDF: 109
- XML: 26
- Total: 400
- BibTeX: 15
- EndNote: 23
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
1. The authors state that LSM3 slightly outperforms LSM2. However, from a theoretical perspective, weighted ordinary least squares (LSM2) is mathematically equivalent to ordinary least squares with weighted variables (LSM3). Could you please explain why a performance difference is observed in practice?
2. Page 5, Line 156 & Page 6, Line 168: Maybe change ‘.when ...’ to ‘.When...’.
3. Page 6, Eq.(6): The weight factor is defined as a function of transmittance (i.e., W ∝ τ). Is this derived from an iterative optimization procedure, or other physical assumptions? Could you please add a description to clarify the principle behind this specific choice?
4. Page7, Lines 202-214 & Lines 215-220: This part needs to be carefully revised. It appears to be a repetition of content, where two consecutive paragraphs convey essentially the same meaning with slightly different wording.
5. Page 13, Line 331-332: The authors used a semi-empirical threshold of 10-4 as the screening criterion. However, it is not entirely clear whether this threshold is specifically selected for the FY-3F HIRAS-II instrument based on dedicated analysis, or adopted from previous studies as a generally applicable value. Could you please provide more quantitative evidence to support this choice if possible?
6. Figure quality: Most of the comparison figures (e.g., Figs. 5, 6, etc.) appear noticeably blurred and contain more than 15*3 lines in a single image, making it difficult for readers to distinguish between different schemes and symbols. To improve readability, I suggest select a limited number of representative profiles (e.g., Profiles 81, 82, 83) to illustrate the key findings, and move others to the appendix or supplementary material.
7. Figure caption: Different methods are described using words in Fig. 5 (e.g., ‘pentagrams’, ‘circles’, ‘triangles’), while symbols in Fig. 9 (e.g., ‘✯’, ‘○’, ‘△’). For reasons of consistency, please unify the notation style (i.e., either textual descriptions or symbols).