the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 24 Jan 2025
The Impact of Differences in Retrieval Algorithms between Processing Centers on GNSS Radio Occultation Refractivity Retrievals in the Planetary Boundary Layer
Abstract. GNSS radio occultation (GNSS RO) performance in the planetary boundary layer is strongly dependent on retrieval algorithms. In this work, we explore how differences in retrieval methodology across three major processing centers of GNSS RO data — NASA JPL, ROM SAF, and UCAR — impact refractivity retrievals in the planetary boundary layer. Using a shared base of occultations from the FORMOSAT-3/COSMIC-1 GNSS RO mission, we identify key differences between the three processing centers that are especially strong in the regions of frequent super-refraction. We find that the minimum penetration height allowed by each processing center is correlated with the amount of super-refraction, resulting in poorer penetration and higher refractivity biases in the Tropics. We found JPL to have the most conservative minimum height in this region at 1 km, followed by ROM SAF (640 m), and UCAR (420 km). We identify two key geopotential heights — 0.8 km and 2.6 km — to sample the global distribution of inter-center refractivity bias, finding differences of 0.3–0.5 % in the Tropics. We also find negative refractivity biases of up to -4 % relative to ERA5 reanalysis in regions of persistent high stratocumulus coverage, and areas along the descending branch of the Hadley circulation, with negligible bias along the intertropical convergence zone. A comparison to ERA5 also reveals areas of weak (0.2–0.5 %) positive refractivity biases in polar regions. We hypothesize potential causes for these biases based on truncation schemes, radio-holographic filtering choices, and quality control, and identify findings deserving of further investigation.
- Preprint
(1579 KB) - Metadata XML
- BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on egusphere-2024-4127', Anonymous Referee #1, 20 Feb 2025
The authors perform a statistical comparison of radio occultation retrievals of UCAR, ROM SAF, and JPL. It is demonstrated that the differences between the three processing centers are strongest in the regions of frequent super-refraction. The authors point out that the minimum penetration
height allowed by each center correlates with the super-refraction strength. The study reveals negative refractivity biases of up to −4% wrt to ERA5 reanalysis in regions of persistent high stratocumulus coverage. This makes the paper an interesting contribution.On the negative side, the paper misses important points. The abstract states: "In this work, we explore how differences in retrieval methodology across three major processing centers of GNSS RO data—NASA JPL, ROM SAF, and UCAR—impact refractivity retrievals in the planetary boundary layer". However, actually, the analysis of the "differences in retrieval methodology" is superficial. Surprisingly, the authors do not discuss the ionospheric correction and statistical optimization algorithms. These have many possible implementations with many tunable parameters. Their choice differs between processing centers and is the primary candidate for the reason of discrepancies. The authors make a series of statements without references and argumentation. Some statements are imprecise although the authors are aware of the references with a clear analysis. One of the consequences of this is a vague language. The conclusion of this reviewer is that the paper can be published after a major revision.
L.8. ...and UCAR (420 km).-- 420 m.
L. 36. Downward vertical gradients in the microwave index of refraction can become so strong in a layer of the atmosphere that rays are ducted, rendering the ducting layer invisible to external rays, such as those from the GNSS transmitter: a phenomenon known as super-refraction. RO can measure the atmosphere above and below such ducts but never inside them.-- The inverse problem in the presence of ducts has no unique solution, and this affects also the profile below the duct. I suggest putting this in stricter terms, along the lines of the references:
1. Sokolovskiy, S. V. (2003), 'Effect of super refraction on inversions of radio occultation signals in the lower troposphere', Radio Sci. 38(3), 1058. DOI: 10.1029/2002RS002728
2. Sokolovskiy, S., Schreiner, W., Zeng, Z., Hunt, D., Lin, Y.-C. and Kuo, Y.-H. (2014), 'Observation, analysis, and modeling of deep radio occultation signals: Effects of tropospheric ducts and interfering signals', Radio Sci. 49(10), 954–970. DOI: 10.1002/2014RS005436
3. Sokolovskiy, S., Zeng, Z., Hunt, D. C., Weiss, J.-P., Braun, J. J., Schreiner, W. S., Anthes, R. A., Kuo, Y.-H., Zhang, H., Lenschow, D. H. and Vanhove, T. (2024), 'Detection of Superrefraction at the Top of the Atmospheric Boundary Layer from COSMIC-2 Radio Occultations', J. Atmos. Ocean. Technol. 41(1), 65–78. DOI: 10.1175/jtech-d-22-0100.1
4. Xie, F., Syndergaard, S., Kursinski, E. R. and Herman, B. (2006), 'An Approach for Retrieving Marine Boundary Layer Refractivity from GPS Occultation Data in the Presence of Superrefraction', J. Atmos. Ocean. Technol. 23(12), 1629–1644.
L. 62. The generation 3 missions, including the six satellites of COSMIC-2 and the satellites of the commercial RO provider PlanetiQ, obtain median SNRs of roughly 2000 V/V (1 Hz). These new, exceptionally large SNRs allow RO signal tracking deeper into the PBL than ever before, even in the presence of extreme bending and super-refraction.-- See the discussion of the SNR:
5. Gorbunov M., Irisov V., and Rocken C. Noise Floor and Signal-to-Noise Ratio of Radio Occultation Observations: A Cross-Mission Statistical Comparison. Remote Sensing. 2022, 14(3), 691, DOI: 10.3390/rs14030691
6. Gorbunov M., Irisov V., and Rocken C. The Influence of the Signal-to-Noise Ratio upon Radio Occultation Retrievals, Remote Sensing. 2022, 14(12), 2742; DOI: 10.3390/rs14122742.
L.65. Very recent results show that these high-SNR RO soundings enable the detection of the presence of super-refraction (Sokolovskiy et al., 2024) and the critical refractional radius1 that defines the super-refraction duct, so long as the layers are not attached to Earth’s surface (Zeng et al., 2024).-- What exactly does it mean: "not attached to Earth’s surface"? Do you want to say that we only know the refractive radius rather than the geometric height of the duct?
L. 68. UCAR has recently begun publishing level 1 (calibrated excess phase) COSMIC-2 RO data with a super-refraction detection flag and a value for the duct refractional radius; those data can then be used in a retrieval of water vapor from RO data that seeks to be unbiased.-- Any references regarding the use of these data for water vapor retrieval?
L. 71. While unbiased retrievals of water vapor in the PBL from RO data have yet to be published, the existing retrieval algorithms can be examined for systematic and structural refractivity errors in the PBL.-- There are no unbiased retrieval algorithms? Or they are under development, but not yet published? Provide here references on the existing water vapor retrieval algorithms. Use more precise language.
L. 72. The components of existing retrieval systems that can induce bias include implementations of navigation message demodulation, radio-holographic filtering and the smoothing it begets, the wave optics retrieval itself, and the approach to cutting off an RO signal low in the atmosphere when the signal
becomes too weak to be of use.-- Provide here the references on the biases from each item: 1) radio-holographic filtering; 2) the wave optics retrieval; 3) cutting off.
L. 91. Their [FSI, CT2, PM] performances are similar but not identical.-- Provide references. My experience tells that the bias is mostly defined by the aforementioned implementation options rather than by the choice of a specific FIO-based algorithm.
L. 160. While, in principle, Eq. 1 is integrated up to an impact parameter of infinity, in reality each processing center must make a choice of what maximum altitude to integrate up to and handle contributions due to layers higher in the atmosphere with great care. Typically, this is somewhere in the thermosphere.-- The authors write once again the Abel transform pair, which can be found in hundreds of publications. The whole paper does not say a word about ionospheric correction and statistical optimization! These algorithms, depending on many parameters and options, constitute one of the most important causes of cross-center differences!
L. 175. UCAR uses a phase matching technique (Kuo et al., 2004; Sokolovskiy, 2001, 2003; Sokolovskiy et al., 2010), while JPL (Hajj et al., 2002) and ROM SAF use a type-2 canonical transform (Schwärz et al., 2024; Syndergaard et al., 2020, 2021). These techniques produce similar but not identical results, and will produce refractivity retrieval differences systematic to each processing center.-- The differences are not so much due to different FIO algorithms, but due to the other details. See above.
Correct the URLs of the references.
Syndergaard et al., 2020:
https://rom-saf.eumetsat.int/product_documents/romsaf_atbd_ba.pdf
Syndergaard et al., 2021:
https://rom-saf.eumetsat.int/product_documents/romsaf_atbd_ref.pdf
L. 178. This effect is especially strong around a geopotential height of 2–3 km.-- Once you speak about a roughly approximate height, it is totally unnecessary to distinguish between the geometrical and geopotential height, which have much smaller differences especially at 2-3 km. Note, the geopotential height is measured in gpkm.
L. 185. Furthermore, quality control and choice of minimum height create retrieval differences between processing centers. Quality control measures discard occultations at any of the three processing steps that produce sufficiently anomalous occultations. JPL, for example, discards refractivity retrievals with a refractivity difference between RO and that predicted by a reanalysis model greater than 10%. What constitutes “anomalous” is determined by each processing center and would be expected to produce additional systematic sampling biases.-- The choice of the minimum height is often performed dynamically based on the amplitude after the FIO processing. Mention this, too. Write more about QC and criteria of events, "sufficiently anomalous" to be rejected. Are there any studies of the sampling differences that "would be expected"?
L. 200. This creates a small correlation between RO retrievals, especially from UCAR, and the reanalysis.-- Any qualitative estimates of the "small" correlation?
L. 211. ... only of only ...
L. 216. Differences in quality control between different processing centers at different stages—from the calibrated phase vs. time (level 1a data), to bending angle vs. impact parameter (level lb), to refractivity retrieval vs. geopotential height (level 2)—each step in processing includes checks to discard anomalous occultations. These checks may include, among other strategies, comparison of retrievals with models or using noise thresholds.-- Any references describing different QC procedures?
L. 221. The sage green center region in the center of the figure shows that 67.2% of the occultations were processed by all three centers. This indicates a strong level of agreement in quality control.-- You have not formulated any criteria for "strong" of "weak" level of agreement. Therefore, the second sentence is unnecessary. Just say it is 67.2%.
L. 223. The higher processing rate of UCAR may be a result of COSMIC-1 being a UCAR-affiliated mission. ROM SAF and UCAR share calibrated phase data; which may explain why they share a higher percentage of retrievals not processed by JPL (15.4%). This indicates a difference in quality control at between UCAR and JPL at the calibrated phase level.-- How do we know that there is a QC difference at the calibrated phase level? Obviously, a stricter QC at any processing stage will result in a higher rejection rate. Remove "at" in "in quality control at between".
L. 229. The minimum height allowed by retrievals is a critical characteristic of quality control in RO processing. As a ray penetrates deep into the atmosphere, effects such as super-refraction layers, atmospheric multi-path, topography, and code demodulation all cause a decrease in SNR. Retrieval algorithms must therefore make a choice to truncate the occultation at some minimum altitude.-- Any references describing different cut-off procedures? E.g. cut-off from straight-line tangent altitude, cut-off from CT amplitude etc. Do not replace specific analysis by obvious statements like this: "Each processing center makes different choices about the parameters determining how deep an occultation is allowed to penetrate". Note, there is a trade-off between the penetration depth and retrieval accuracy. It would be interesting to complements Figure 2 with a comparison of retrieval with, e.g. ECMWF re-analyses.
L. 271. The band of red stippling near the equator in each plot approximates the location of the intertropical convergence zone (ITCZ). We find that the band of lower minimum height (better penetration) roughly tracks the ITCZ, while the highest minimum heights are in the subsiding branches of the Hadley cells.We hypothesize that subsidence produces sharp refractivity gradients that result in a bias towards higher minimum heights (poorer penetration), while convection along the 275 ITCZ reduces the vertical discontinuities in refractivity, causing quality control measures to allow processing of occultations deeper into the PBL.-- I don't see much correlation between red stipplings and penetration. The above statements are not confirmed by any qualitative analysis nor references. Either present more quantitative analysis confirming your statement, or remove the red stipplings.
L. 301. The vertical profile of the rightmost panel also shows that the positive bias of ROM SAF is very large in the Tropics at 2.6 km and shrinking quickly down to 0.8 km, so the positive bias of ROM SAF in the Tropics may really be more indicative of the strength of the positive bias at higher geopotential heights.-- I don't understand the last part of this statement. Why should the bias at 2.6 km be indicative of bias at larger heights? Why not simply plot the bias at larger heights?
Citation: https://doi.org/10.5194/egusphere-2024-4127-RC1 -
RC2: 'Comment on egusphere-2024-4127', Anonymous Referee #2, 05 Mar 2025
This is an interesting manuscript using an AWS data repository. If published, one could also provide the plotting commands in a Jupyter notebook for others to use.
However, I do think that this work needs more substantial improvements, as it stands now, it is "just another structural uncertainty" investigation, that does not go deeper down into the actual causes. The found differences are very likely due to different processing and data screening implementations. When the reprocessing was done, and from what level (e.g. from level 1a, or level 0) will also play a role. More information on how that data screening and particular processing is implemented at the different centers should be included. Furthermore, it is highly recommended to include a comparison with respect to a reference, thus guiding the processing centres on how well their implementations work.More specific commands with page/line are below:
- P1L8: 420m I assume
- P2L49: The local spherical symmetry limitation has been partly corrected, at least at ECMWF, by ray tracing through the lowest atmospheric layers.
- P3L64: "These new ..." So far, several low SNR missions, e.g. like Spire, have also shown good penetration into the PBL. I think it is not as much a penetration problem, but rather a problem of how much of the bending to very low SLTA values is captured by the RO instrument. Without capturing this deep signal, the penetration can still get into the PBL, but the retrieved profiles are more biased as they miss this deep part.
- P4L95: The cut off can also be performed in time (looking at SRN), or SLTA. I believe this is the more common approach, so that the WO algorithm is not operating on all data. But maybe I misunderstood your statement.
- Section 2: I don't think, we need to go through the basics of RO here. It has been published often enough. Section 2 can thus be shortened to focus on the relevant.
- P5L133: Why is C2 here 1500V/V, and above 2000V/V?
- P7L194: My last comment on UCAR reference location - maybe mention it here, as this is easily over read or misunderstood where the ERA5 data actually came from.
- Eq 3: This is the rather old, and by now slightly modified equation for refractivity. Did you really use this, or did you use more recent versions, as e.g. also used in the ROPP tool?
- Figure 1: Is there a possibility to use more distinctive colors? Not sure I know what lime green is vs. sage green. You could even leave the centre white. On second thought, I don't see the need for this Venn diagram, a table would also do. There, you could even provide further info, like total number of occs, setting vs rising, etc.
-
P8L223: The found differences in occultations processed might be due to which version is available in the AWS repository. I think, you need to provide some more info there, as e.g. the ROM SAF data might come from their last reprocessing, which used an older UCAR atmPhs data set, while the UCAR data might already be the most recent (I believe 2021) reprocessing.
Also, more generally, processing of RO data improves continuously, thus differences found here could be due to when the processing system was actually set up. The ROM SAF is going to release an updated COSMIC-1 reprocessing in the near future, which might than be more similar to what UCAR did.
- P8L226: Are these 2% from one particular GPS satellite? Or in any other way having a common characteristic?
- P8L224: remove at?
- Figure 4: when zooming in, it seems that pixels overlap with land. I assume this is only the plotting, correct? So if the pixel has even only a small ocean part, it is plotted, but only the ocean occs are included? How do you account for orography near the coast? And, given that the ocean part of that pixel is rather small, you'd also have only few occultations that count towards the median - thus, these pixels could have higher noise values.
- P11L273: no need to hypothesize here, that has been shown various times, maybe just include an appropriate reference.
- P11L280: Interpolation linear in log space? As REF varies essentially like pressure, exponentially?
- P13L297: Did you also apply this minimum for the Figure 4 map?
- P15L320: BTW, do you do any sea ice screening? Could reflections, entering the retrieval scheme, cause these differences?
- Section 3.5: I am unsure why ENSO should have an impact, globally. Any reason for looking into this separately? And, maybe just mention in a sentence the non existing ENSO impact, instead of adding a figure that reveals no info.
- P16L345: ... is hightest *in*? the subsiding...
- P18L370: ... the the ...
- P18L387: You used the same ERA5 profiles for all occultations? An issue here is that UCAR uses a different reference tangent point than e.g. the ROM SAF. So the ERA5 UCAR profiles could be more than 100km off to where ROM SAF puts the reference - but at least this is consistent in your use (we right now have a C2 discussion where some of the biases found are due to the different reference tangent points). Maybe mention that higher up.
Citation: https://doi.org/10.5194/egusphere-2024-4127-RC2
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 124 | 24 | 7 | 155 | 5 | 6 |
- HTML: 124
- PDF: 24
- XML: 7
- Total: 155
- BibTeX: 5
- EndNote: 6
Viewed (geographical distribution)
| Country | # | Views | % |
|---|---|---|---|
| United States of America | 1 | 83 | 52 |
| China | 2 | 11 | 6 |
| Germany | 3 | 10 | 6 |
| France | 4 | 9 | 5 |
| India | 5 | 6 | 3 |
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
- 83
