the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 10 Dec 2025
An update to the expression of atmospheric refractivity for GNSS signals
Abstract. This study revisits previous formulations of atmospheric refractivity at L-band frequencies, focusing on signals from Global Navigation Satellite Systems (GNSS). A refined model expression is proposed as a function of air density, temperature, and composition, evaluated using a comprehensive set of existing laboratory and atmospheric measurements. The key measurements that most affect the final accuracy are identified, establishing traceable error bounds and indicating where further experimental work could confirm or improve the model.
Recent studies on the use of large volumes of GNSS radio occultation (GNSSRO) observations in Numerical Weather Prediction (NWP) show that the precise formulation of refractivity becomes increasingly critical as data volumes grow. Although the revision is modest, its impact lies within the range where NWP sensitivity becomes non-negligible.
Compared to earlier work, this study (1) incorporates updated fundamental measurements, (2) accounts for the small but measurable variability in atmospheric composition, mainly increasing CO2 and decreasing O2, emphasizing that refractivity traceability is composition-dependent, and (3) extends the model to include hydrometeors. A simplified formulation based on hydrometeor oblateness is proposed, suitable for NWP applications where only limited hydrometeor information is available. Nonspherical hydrometeors tend to align during fall, introducing weak birefringence that can be detected during GNSS occultations with dual-polarization receivers.
The resulting refractivity expression is presented as a function of air density, temperature, moisture, and composition, and (using a simplified model of atmospheric evolution) also as a function of density, temperature, moisture, and time.
- Preprint
(615 KB) - Metadata XML
- BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on egusphere-2025-5986', Anonymous Referee #1, 08 Jan 2026
-
AC1: 'Reply on RC1', Josep M. Aparicio, 20 Mar 2026
“A careful analysis establishes that this target accuracy is achieved, even with the additional effects considered such as composition and hydrometeors. This reviewer suggests updating the abstract with the stated accuracy goal and that it is achieved with the revised expressions”.
The abstract had been updated to reflect this: “It is concluded that the result meets the target accuracy of 0.01%, with all remaining uncertainty sources below this threshold. With respect to earlier work, composition and hydrometeor additions do not modify the largest part of the bulk refractivity above this threshold”.
“Line 66: this paragraph suggests that biases detected by NWP centers using ROMEX data are solely due to the refractivity formulation used. While the refractivity formula probably plays a role, is it justified to ascribe all the biases seen in ROMEX experiments to that formula? I believe this is not the generally accepted view”.
It was not the intent to suggest that. Rather, the intent was, among all potential sources of observation minus background (OMB) bias at ROMEX-level data volumes, to help rule out the refractivity equation as one of the significant sources.
This paragraph and the following, that state an absence of significant bias using this or AL11 “within ECCC’s NWP system” have been modified, and a sentence has been added to indicate that within ECCC, refractivity (AL11 or this work), coupled with a suitable equation of state, is not currently viewed as a source of concern towards OMB RO bias, while it had been in the past (AP09, AL15)."Line 169: this formula references AL11 which does not include the gas phase term $\epsilon^r_g$. How is it decided which of the epsilons in AL11 equation (2) becomes the term for the gas phase?"
AL11 has a single phase, which is gas, whose permittivity is denoted as $\epsilon^r$. In the present model, a given molecule may be immersed in gas $\epsilon^r_g$ (closest to AL11), liquid, ice, or in fact in an effective medium with several phases $\epsilon^r$. In AL11 it was not necessary to consider more than one permittivity, nor to distinguish between single-phase and the effective permittivity of a multiphase mixture.
"Line 274: it would be very useful to have a quantitative upper limit for hydrometeor concentration consistent with the accuracy goals of the paper. Maximum values are listed near line 550. Presumably under conditions when these maxima are reached, the formula does not achieve 0.01% accuracy? If possible, an upper limit of hydrometeor concentrations consistent with 0.01% accuracy would be useful to have in the paper."
Indeed. The main source of uncertainty here is the highly variable hydrometeor concentration and shape under minimally realistic conditions. A sentence in the final comments has been added to indicate that when hydrometeors are present, one should expect an uncertainty of a fraction of the hydrometeor refractivity, at least from the highly variable amount and shape.
"Line 288: “canting” is a somewhat specialized term in this context so please briefly define it."
A short definition has been added.
"Line 567: 200 m difference in altitude is significant, especially since papers have claimed to determine PBL height to approximately this level of precision. Could the birefringence affect estimates of PBL height?"
Perhaps, to some extent. The 200m is a rather extreme case, derived from large but still realistic hydrometeor (rain) density, sustained over a large geographic area (quite infrequent for heavy rain). Nevertheless, a smaller but detectable value would be more common, perhaps a few 10s of meters. Besides, this is the split between both linear polarizations. A dual linearly polarized antenna may see the PBL in the same location, but with a small time difference. A standard circularly polarized antenna will instead see a PBL blurred by the split. I have not yet done any search for this signature on real data, but I may expect at least a distortion of the PBL profile, and perhaps some displacement. Although I would expect this to be small compared to the dependence of the determination of the PBL on the algorithm used.
"Line 570: the descent rate slows considerably in the lower troposphere. Is this the appropriate descent rate in the PBL?"
Indeed, the ascent/descent rate slows down in the low troposphere, including PBL. But given the large uncertainty in hydrometeor amount and shape, the intent was just to provide an order-of-magnitude estimate, to evaluate whether this split is negligible or observable. The conclusion is that it is small but should be observable. The reduction in ascent/descent rate helps the detectability (seeing the PBL at different moments in both polarizations), but ultimately a case-by-case estimate is needed.
"Line 578: please be explicit here that “useful” means accurate to 0.01%, if that is what is meant."
It is intended to mean “sufficient to not be concerned by the remaining uncertainty, at present”. In practice, 0.01% happens to be sufficient for that purpose, whereas 0.1% is not. The text has been revised to clarify.
"Line 724: this paragraph is a little confusing. The threshold of 0.1% is met? (rather than exceeded?). The second sentence is hard to understand. What is meant by “margin”? Existing formulations certainly do not meet 0.01%, even in the number of significant digits retained for the constants (in some cases)."
The 0.1% is exceeded, yes. The 0.1% is the minimum threshold that needs to be exceed. Failing to exceed it, nothing new would be added. The “margin” is the extra accuracy beyond the minimum threshold. The text has been adapted to better convey this.
"Technical Corrections:
Line 177: remove extra “of”"Ok.
"Line 385: remove extra “is”"
Ok.
"Line 431: should this be Eq (18)?"
Yes.
"Line 576: remove extra “have”"
Ok.
"Line 596: should be (28)-(38)?"
Yes, correct.
"Line 635: should be “dependent”, and remove extra “be”"
Ok.
"Line 640: remove “po”"
Ok.
"Line 650: should be “carries”"
Ok.
Citation: https://doi.org/10.5194/egusphere-2025-5986-AC1
-
AC1: 'Reply on RC1', Josep M. Aparicio, 20 Mar 2026
-
RC2: 'Comment on egusphere-2025-5986', Anonymous Referee #2, 16 Feb 2026
An update to the expression of atmospheric refractivity for GNSS signals
By J M Aparicio
General comments
I think this paper will be a very useful contribution to the GNSS radio occultation literature and I recommend publish, subject to the minor clarifications/modifications given below.
There is uncertainty within the GNSS-RO community regarding the best formulation of the refractivity equation for NWP applications. This has relevance to the ROMEX experiments. This paper will certainly help inform the discussions.
Specific comments
Page 3, line 70: I think you are saying that ECCC did not have the same ROMEX bias problems as other centres because of the coefficients, but I think you include other changes such as a non-ideal equation of state when computing the hydrostatic equation. Please list other ECCC differences you think may be relevant to ROMEX biases and clarify the text. Also consider noting that the terms relating to the dry contribution are probably most important for ROMEX, because of the importance of this contribution in GNSS-RO core region, where the data is given most weight.
Equations 17-21. It would be useful to state your assumed equation of state here. Based on the author’s previous work, I assume it is but it should be clearly stated. I would also add the derivation of the density terms from this equation for clarity. This would help users implement the new approach.
Page 20, line 559. . I do not recognise this figure. The total differential delays quoted in PRO papers are usually around 10-30 mm, so it looks to be wrong by a factor of 10. Please clarify.
Figure 2 and line 567: My understanding is most PRO studies assume that the ray path is the same for the two polarisations. Is your analysis consistent with the bending angle differences shown here by Wang et al: https://journals.ametsoc.org/view/journals/atot/39/2/JTECH-D-21-0032.1.pdf
See their Figure 4: 10 microradian differences. The 200 m in the vertical seems much larger bending angle difference. Please explain.
Figure 3 and description on pages 25-26. Figure 3 is very important and I think the reader would benefit from a more detailed description of what is done here. My understanding is you integrate the hydrostatic equation, based on two surface temperature values and a surface pressure value, which I assume is close to 1013.25 hPa. The upper air lapse rates are assumed to be the US standard atmosphere and the atmosphere is dry. This provides P and T as function of height (geometric or geopotential, hydrostatic integrated assuming ideal gas?). You then compute the density from , including the compressibility, , which is a function of pressure and temperature . The density is used in both the new and AL11 curves to compute refractivity. The TH74 curves use P and T in equation 43 to compute refractivity. You then multiply the refractivity values by to obtain and effective for those conditions.
How big is Z at 30 km? Why do TH74 curves not converge to 77.6 at 30 km? Why do the curves for the new expression and AL11 not converge at 30 km, but the TH74 curves do converge?
Aside, for discussion: since the author has probably thought about this subject more than anyone, has he any idea what RU02 got wrong? It seems surprising SW53 is more accurate.
Line 676: GPSRO -> GNSSRO
Line 724-725: Sentence containing “threshold of 0.1 % … is exceeded …”. I think you mean the 0.1 % target is already met when non-ideal gases are included, but this is not very clear in the text. Please clarify.
Line 740: In applications, consider the processing of altimeter sea-surface-height information as an application of this work. They apply a dry delay correction using 77.6, and I think they assume this is a constant.
Section 4 https://www.sciencedirect.com/science/article/pii/S0034425720305228
Citation: https://doi.org/10.5194/egusphere-2025-5986-RC2 -
AC2: 'Reply on RC2', Josep M. Aparicio, 20 Mar 2026
"I think this paper will be a very useful contribution to the GNSS radio occultation literature and I recommend publish, subject to the minor clarifications/modifications given below.
There is uncertainty within the GNSS-RO community regarding the best formulation of the refractivity equation for NWP applications. This has relevance to the ROMEX experiments. This paper will certainly help inform the discussions.
Specific comments
Page 3, line 70: I think you are saying that ECCC did not have the same ROMEX bias problems as other centres because of the coefficients, but I think you include other changes such as a non-ideal equation of state when computing the hydrostatic equation. Please list other ECCC differences you think may be relevant to ROMEX biases and clarify the text. Also consider noting that the terms relating to the dry contribution are probably most important for ROMEX, because of the importance of this contribution in GNSS-RO core region, where the data is given most weight."Indeed. The intent was to mention that at ECCC, OMB biases with AL11 and with this formulation are not concerning, but certainly there are other potential sources of bias. We try to list all sources that may be relevant, and that it is within this context, that the constitutive relationships (mostly refractivity and equation of state) are not viewed as a source of bias.
"Equations 17-21. It would be useful to state your assumed equation of state here. Based on the author’s previous work, I assume it is but it should be clearly stated. I would also add the derivation of the density terms from this equation for clarity. This would help users implement the new approach."
Indeed. It is Picard 2008. It is now mentioned. Please note that the equation of state (EOS) is relevant within the framework of an application, for instance meteorological data assimilation. The refractivity model presented, as a function of density, is prepared precisely to be independent of the EOS, and in fact to emphasize to users the impact of the EOS. The text has been updated in the comments.
"Page 20, line 559. . I do not recognise this figure. The total differential delays quoted in PRO papers are usually around 10-30 mm, so it looks to be wrong by a factor of 10. Please clarify."
This was intended as an estimation of an extreme large case, at the strongest amount of rain, and sustained as a spherically symmetric region over about 50 km. Particularly this latest aspect (a large path of consistently heavy rain) makes it somewhat unrealistic. A sentence has been added noting that this is an extreme upper bound, and that actual rain tends not to be both heavy and geographically wide at the same time.
"Figure 2 and line 567: My understanding is most PRO studies assume that the ray path is the same for the two polarisations. Is your analysis consistent with the bending angle differences shown here by Wang et al: https://journals.ametsoc.org/view/journals/atot/39/2/JTECH-D-21-0032.1.pdf
See their Figure 4: 10 microradian differences. The 200 m in the vertical seems much larger bending angle difference. Please explain. "Indeed, in most cases, the split between paths, either in time or space, is very small, even with measurable differential path. It is only at the highest differential path delay that the split may be significant. Thus the “equal path” is justified for most purposes. The intent here was to link a physical property (birefringence) with a derived geometric property (split of the propagation path) that is small but not unobservable.
The 200m in geometric (not impact) height (which would be a few 10s of microrads from a typical distance of 3000 km) is an extreme case, as mentioned above (this is further emphasized in the text). Reasonably frequent cases should be a bit smaller, thus 10 microrad would be consistent of a more frequent, rather than extreme case.
"Figure 3 and description on pages 25-26. Figure 3 is very important and I think the reader would benefit from a more detailed description of what is done here. My understanding is you integrate the hydrostatic equation, based on two surface temperature values and a surface pressure value, which I assume is close to 1013.25 hPa. The upper air lapse rates are assumed to be the US standard atmosphere and the atmosphere is dry. This provides P and T as function of height (geometric or geopotential, hydrostatic integrated assuming ideal gas?). You then compute the density from , including the compressibility, , which is a function of pressure and temperature . The density is used in both the new and AL11 curves to compute refractivity. The TH74 curves use P and T in equation 43 to compute refractivity. You then multiply the refractivity values by to obtain and effective for those conditions. "Correct.
"How big is Z at 30 km? Why do TH74 curves not converge to 77.6 at 30 km? Why do the curves for the new expression and AL11 not converge at 30 km, but the TH74 curves do converge?
Z does converge towards 1 with altitude. TH74 uses its own estimation of Z, separately for dry air (Zd) and water vapour (Zw), whereas this work rather choses Picard 2008. TH74 does converge to 77.6."Both become more ideal at lower pressure/density. 1-Z at 30km is about 3e-5, whereas it can be 1e-3 at surface. Picard 2008 represents differently the departure of ideal behavior with low temperature (thus stratosphere), and is slower to approach ideal behavior. But more importantly, and notably above 20 km, the shape of the effective “k1” (N*T/P) in AL11 and here is affected in typical atmospheric conditions by a temperature dependence of the dry term (the coefficient q2, from O2 paramagnetism: O2 is polar to magnetic fields, whereas water is polar to electric fields). Whereas q1 and Z would lead NT/P to converge to a constant at low pressure, the q2 does not. In the comments, a few words are added regarding the stratospheric behaviour.
"Aside, for discussion: since the author has probably thought about this subject more than anyone, has he any idea what RU02 got wrong? It seems surprising SW53 is more accurate. "RU02 was intended for low altitude (horizontal distance measurement), and correctly absorbs an average (constant) low-altitude Z. TH74, AL11 and this model all suggest a large effective k1 at low altitude, which is similar to RU02. Thus, RU02 was not a bad choice for Rueger’s intended purpose.
For RO, on one hand, the most impacting region is (300-50 hPa), and the average assumed Z absorbed in RU02’s coefficients is not very representative of this region, thus it was inappropriate to use RU02 above the low troposphere. On the other hand, it is particularly inaccurate to use RU02 without also applying Z to the hydrostatic equation.
Within this context, SW53 is not very accurate at low altitude, but happens to be better if applied at higher altitude AND assuming that air is ideal, which is the simplest approach in NWP. Thus, (ideal gas+SW53) happens to be close to optimal among simple choices for RO. Beyond this combination, the next best is to consider a realistic compressibility, but must be done consistently in both the refractivity and in the hydrostatic equation."Line 676: GPSRO -> GNSSRO"
Ok
"Line 724-725: Sentence containing “threshold of 0.1 % … is exceeded …”. I think you mean the 0.1 % target is already met when non-ideal gases are included, but this is not very clear in the text. Please clarify. "
Ok
"Line 740: In applications, consider the processing of altimeter sea-surface-height information as an application of this work. They apply a dry delay correction using 77.6, and I think they assume this is a constant.
Section 4 https://www.sciencedirect.com/science/article/pii/S0034425720305228"This application, as well as ZTD, has been added.
Citation: https://doi.org/10.5194/egusphere-2025-5986-AC2
-
AC2: 'Reply on RC2', Josep M. Aparicio, 20 Mar 2026
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 288 | 197 | 32 | 517 | 17 | 17 |
- HTML: 288
- PDF: 197
- XML: 32
- Total: 517
- BibTeX: 17
- EndNote: 17
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
General Comments:
The paper is a welcome addition to the literature. It builds on a previous work, Aparicio and Laroche (2011) (AL11), and adds significant new elements including accounting for variation in atmospheric composition (CO2, O2) over time, and the refractivity contribution of hydrometeors. For the first time to this reviewer’s knowledge, the paper discusses the birefringence of the atmosphere in the presence of hydrometeors and in the context of radio occultation (RO), providing useful estimates of the how the propagation path varies depending on polarization. This paper provides traceability from the uncertainties of the fundamental measurements of microphysical properties to uncertainty of the expression for refractivity, which is a significant new contribution. The paper is clear and concise.
The target accuracy for the refractivity expression is 0.01%, the same as the accuracy claim in AL11. A careful analysis establishes that this target accuracy is achieved, even with the additional effects considered such as composition and hydrometeors. This reviewer suggests updating the abstract with the stated accuracy goal and that it is achieved with the revised expressions.
Specific Comments:
Abstract: as noted above, the abstract would be improved if the stated accuracy goal of 0.01% is mentioned. Also useful to mention is that the new analysis is consistent with this goal.
Line 66: this paragraph suggests that biases detected by NWP centers using ROMEX data are solely due to the refractivity formulation used. While the refractivity formula probably plays a role, is it justified to ascribe all the biases seen in ROMEX experiments to that formula? I believe this is not the generally accepted view.
Line 169: this formula references AL11 which does not include the gas phase term $\epsilon^r_g$. How is it decided which of the epsilons in AL11 equation (2) becomes the term for the gas phase?
Line 274: it would be very useful to have a quantitative upper limit for hydrometeor concentration consistent with the accuracy goals of the paper. Maximum values are listed near line 550. Presumably under conditions when these maxima are reached, the formula does not achieve 0.01% accuracy? If possible, an upper limit of hydrometeor concentrations consistent with 0.01% accuracy would be useful to have in the paper.
Line 288: “canting” is a somewhat specialized term in this context so please briefly define it.
Line 567: 200 m difference in altitude is significant, especially since papers have claimed to determine PBL height to approximately this level of precision. Could the birefringence affect estimates of PBL height?
Line 570: the descent rate slows considerably in the lower troposphere. Is this the appropriate descent rate in the PBL?
Line 578: please be explicit here that “useful” means accurate to 0.01%, if that is what is meant.
Line 724: this paragraph is a little confusing. The threshold of 0.1% is met? (rather than exceeded?). The second sentence is hard to understand. What is meant by “margin”? Existing formulations certainly do not meet 0.01%, even in the number of significant digits retained for the constants (in some cases).
Technical Corrections:
Line 177: remove extra “of”
Line 385: remove extra “is”
Line 431: should this be Eq (18)?
Line 576: remove extra “have”
Line 596: should be (28)-(38)?
Line 635: should be “dependent”, and remove extra “be”
Line 640: remove “po”
Line 650: should be “carries”