the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 03 Sep 2025
Experiments with large number of GNSS-RO observations through the ROMEX collaboration in the Met Office NWP system
Abstract. Over recent years there has been an increase in the number of GNSS-RO observations available for use in numerical weather prediction (NWP). The Radio Occultation Modelling Experiment (ROMEX) was set up to assess the impact of increasing numbers of GNSS-RO observations and to provide evidence for a further increase in the number. Unlike previous studies, ROMEX gathered a large set of real observations to test the impact, rather than use simulated observations.
Tests assimilating the ROMEX observations into the Met Office's NWP system showed negative impacts on forecast quality. This was largely due to a bias in the forecasts of geopotential height in the troposphere. This bias is shown to be due to the impact of GNSS-RO observations in the stratosphere. The forward operator for GNSS-RO has a long ``tail'' meaning that the NWP system is able to adjust the height of observations in the stratosphere by changing the tropospheric state (i.e. raising or lowering the height of the observation). Thus, the NWP system can adjust to a systematic difference between the model and observations in the stratosphere by creating a bias in the tropospheric state. Therefore, it was necessary to adjust the refractivity operator used in the forward model to reduce this bias in the forecast state. After much experimentation it was decided to alter the refractivity coefficients in the operator by 0.1 % and 3.5 % for k1 and k2, respectively. When using this adjusted operator it was possible to demonstrate the beneficial effect of assimilating the ROMEX observations.
Additional modifications were also applied to the processing of GNSS-RO observations, including vertical smoothing of the observed profiles, and a bias correction at high altitudes to correct for errors within the NWP model. With this modified operator various experiments were conducted to assess impact of increases to the total number of observations. It was shown that the addition of the extra ROMEX observations provided a substantial improvement in the forecast quality. This is particularly true in the southern-hemisphere extra-tropics where the largest benefits were seen for the additional data. The benefit seen for a given number of additional observations varied substantially with the region being considered and the data against which the verification was being performed. Overall the largest forecast improvements were seen when assimilating 20,000 occultations per day, although this may be connected to the quality of the observations being excluded when reducing the ROMEX dataset to this number.
- Preprint
(3375 KB) - Metadata XML
- BibTeX
- EndNote
Status: final response (author comments only)
- RC1: 'Comment on egusphere-2025-4194', Anonymous Referee #1, 25 Sep 2025
- EC1: 'Replacement of Figure 6 on egusphere-2025-4194', Hui Shao, 25 Sep 2025
-
AC1: 'Comment on egusphere-2025-4194', Neill Bowler, 26 Sep 2025
We have become aware that Figure 16, showing the biases of various observation groups against the NWP model, is affected by an issue in the software which produced the graph. This issue means that the software that produced the graph and produced the biases on which the bias correction scheme was based did not account for the drift of the tangent point in the observations profile. This particularly affects COSMIC-2 due to its low-inclination orbit, making the observation biases appear more positive than when tangent-point drift is accounted for. An updated figure will be posted here as soon as possible.
Citation: https://doi.org/10.5194/egusphere-2025-4194-AC1 -
RC2: 'Comment on egusphere-2025-4194', Anonymous Referee #2, 09 Oct 2025
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2025/egusphere-2025-4194/egusphere-2025-4194-RC2-supplement.pdf
-
RC3: 'Comment on egusphere-2025-4194', Anonymous Referee #3, 13 Oct 2025
The manuscript presents a nice study on the impact of large numbers of real
GNSS-RO data on the MetOffice NWP system in the context of the ROMEX project,
and shows some interesting lessons that can be learned with such a large
dataset. It is well readable and mostly very clear, nevertheless it might
need explanations in some places to address readers outside of the
radio-occultation community or who are not familiar with NWP systems or data
assimilation for NWP.Futhermore the manuscript could be slightly improved by addressing the minor
issues and needed clarifications as discussed below.- Page 2, line 40: CGMS is not a subgroup of WMO, but a "multi-lateral
coordination and cooperation across all meteorological satellite operators
in close coordination with the user community such as WMO, IOC-UNESCO, and
other user entities" (text taken from https://cgms-info.org/about-cgms/)- Page 4, line 90ff: the description of the NWP system could be slightly more
specific. Analyses every 6 hours? Which forward model used for GNSS-RO?
There are several implementations, and technical details like refractivity
expression etc. are of interest. (It seems clear to me that bending angle
is used, but some centres use refractivity.) A reference, ideally also
outlining the modeling of observation error would be great.(Section 2.4 does discuss refractivity in passing.)
Did the authors make initial assumptions about data quality from different
missions? Some NWP centres choose not to assimilate all data equally, and
the authors themselves noted (line 342) that using data from a particular
mission showed a degradation of the scores.Line 91: Not really important, but does the lowest model layer have a
full-level height of 20m, or is it the actual depth of that layer?- Line 102ff: when presenting forecast scores, always specify against which
"truth" the verification is performed. Neither the text nor the caption of
figure 1 explain the observation type being used (radiosondes)?
The caption also refers to surface observations (SYNOP), but I cannot find
related entries in the figure.- Line 116 and figure 2:
- does the figure show the mean error for all observations processed, or
has a quality control been applied?- there are several spikes (or wiggles) of varying amplitude visible in the
plot. What is the origin of these spikes? Interpolation of observations
or of model equivalent? E.g. the region between 15 km and 20 km is so
noisy that the reader can hardly guess the average bias.The spikyness also varies a lot with latitude band (fig.15). Can this
have an effect on the results shown in fig.16?- does this spiky behavior also occur in the forward operator used in the
variational assimilation? If so, does it affect the modeling of error
(background / observation)?- Page 8, line 134: the "golden region" may be colloquially known to experts
in the field, but either a reference or a less colloquial description would
help the non-experts.- Line 148: "This shows a large reduction in the forecasts ..."
The reduction is shown here for *bias*.It appears that geopotential bias is globally averaged (without stating
so). Is this effect seen similarly in the extratropical hemispheres?Regarding the quantification of the reduction: this is against ECMWF
analyses! While it is numerically fine, one could also argue that analysis
biases are more consistent after the adjustment. On the positive side,
the *global drift* of bias during forecast seems much reduced.- Page 15, section 2.5: please specify a "typical" (or reference) lead time
of background forecasts.- Line 21: "... fit to the independent observations ..."
What does "independent" refer to here? Does this express that these
observations were not assimilated, and the set of observations in fig.12
is different from those in fig.11?- Page 25, line 319: "... whereas the ECMWF system has little bias".
Everybody may argue that this is true to a good extent, but some would
rather say "... presumably has a smaller bias", or similar.- Line 320ff: "This may be partly due to the model ..."
I find the description of the treatment of variables in the forward
operator very confusing and not helpful. Could issues in the forward
operator also explain the mean error patterns seen in fig.2/fig.15, or is
there a relation?- Page 26, line 346ff: reference or personal communication?
- Page 26, section 4.2: please specify the background lead time.
- Page 28, line 371ff: the text here and the caption of figs.22-25 are not
consistent. The text refers to a change in RMSE, where the reader expects
a decrease if the systems improves (similarly to fig.21 for the (O-B)
statistics), while the figure captions refer to the "RMSE scorecard", where
an increase denotes better results. Can the authors please resolve this?- Page 30, line 388: "differences in locations of the verifying data points"
It is not the locations alone but data density and spatial sampling that
skews the verification against observations where each profile is both used
and counted, while verification against analyses is less affected. I
recommend to reformulate slightly.Citation: https://doi.org/10.5194/egusphere-2025-4194-RC3 -
RC4: 'Comment on egusphere-2025-4194', Anonymous Referee #4, 16 Oct 2025
This article “Experiments with large number of GNSS-RO observations through the ROMEX collaboration in the Met Office NWP system” by Bowler and Lewis discusses the impact by increasing GNSS-RO observations during ROMEX on numerical weather prediction. First, it was stated that forecast scores were degraded caused by stratospheric effects which required adjustments e.g. to the refractivity operator and data processing. After these corrections, assimilating more GNSS-RO data—particularly around 20,000 observations per day—greatly improved forecast accuracy, especially in the southern-hemisphere extra-tropics.This manuscript highlights nicely the potential but also the challenges coming from assimilating a high number of GNSS-RO data, which wasn’t done before. The authors looked into many different aspect, e.g. bias correction and changes to the forward operator to address the challenges. I recommend publishing these results after major revision.Main points:In general, I think the manuscript needs to be slightly trimmed and restructured perhaps. e.g. the explanation of RMSE/ bias is done after using this measure in various score cards and with using only observations as a reference. Also, many sensitivity studies have been done to tackle the bias in geopotential. Rather showing all the different scorecards and only have a little discussion about them, I would focus on the most important ones but keep mentioning the results obtained from the various experiments with e.g. bias correction. I truly believe this makes the manuscript more readable.Also, I am a bit reluctant to accept the main conclusion that there is a saturation in impact for the 20.000 daily occultations. As the authors discussed, an alternative flavour of this sample showed lower forecast impact, which could cause different fits as previously. This needs to be shown and discussed referring to a possible impact of data quality. However, here I would be careful not to only make the quality of FY3E responsible for that behaviour without making an additional analysis. Nevertheless, differences in timing of the observation, geographical coverage and quality of the observations are playing a key role in their impact.Other pointsMissing brackets for citing other literature throughout the paperP1, abstract: I find it confusing to read first about negative impacts and then later substantial improvement in forecast quality. Please make it clearer what changed to get better forecast with more RO data.P1, l21/22: This sounds like a hypothesis which has not been analysed in this study - hence, I would avoid stating that.P3, l78: typo in observationP4, l91 Would add “horizontal” before “resolution”P 4, Section 2.1: Please state which metric is being looked at. RMSE?P4, l.105: How big is "large". Please quantify.P5, l120-123: What do you mean with adjusting the observations? Modifying the bending angles which are assimilated or the corresponding impact? parameter?P8, l137 – same as in previous comment. What do you increase here?P12, Figure 12: What is pert_all? A perturbation of bending angles in all height levels or an average of pert_5km and so on. If the latter, how can pert_all be positive but the individual impact heights are mostly negative?P12, L178: Which change? The change due to adding ROMEX compared to the control?P 13, l190: “may be preferred” sounds a bit out of place here. Maybe typo?P15, l 200-204: Here or even better earlier it would be good to discuss if this change in the mean/bias is solely or partly responsible for degradations seen in RMSE (scorecards). What about std deviation? Does this also contribute to an increase in RMSE?P15, Eq (3): This would be good to define earlier in the manuscript.P17, Eq(4): I would rename oi,t as truth or reference, which can be observations but in other instances analysis.P18: This part jumps a bit from verifying against observations and against analysis - hence I'd define standard deviation and rmse with using reference or truth rather than observations.P25, l 314: remove one of the “at”sP25, paragraph l 318-324: It would be good to see the impact in forecast scores for that experiment.P26; paragraph l.342-349: This suggests that FY3E caused the degradation. Can this be said or are the results only suggestions it could be due to different "quality"? Looking at Fig 20 I'd said the latter. Could Fig 19 be edited including the alternative 20k sample?P28, l370-371: Is this still valid to say after showing that the alternative sample in the 20k experiment did show a different behaviour? Hence, using the alternative 20k experiment might give you a different answer.P33, l408: Please, mention in which forecast score metric this degradation can be seen and why it was unexpected.P34, l424: I would rephrase this part of the sentence “standard deviation of forecast error was improved”. I’d rather say that compared to the reference the standard deviation decreased, which can be interpreted as an improvement or something similar.Citation: https://doi.org/
10.5194/egusphere-2025-4194-RC4
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 1,775 | 41 | 23 | 1,839 | 15 | 10 |
- HTML: 1,775
- PDF: 41
- XML: 23
- Total: 1,839
- BibTeX: 15
- EndNote: 10
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
The manuscript provides impact assessment of massive GNSS-RO observation on UKMO’s NWP system. The first half of the manuscript discusses the cause and solution of the forecast score degradation due to the introduction of large volumes of data, while the latter half presents how forecast score changes with the increasing number of observations. Observing System Experiments (OSEs) show that simply increasing the number of observations degrades forecast scores especially at troposphere. Sensitivity experiments employing 1D-Var show that observations at lower stratosphere induce change in the troposphere. OSEs with modulated refractivity observation operator coefficients, which aims to reduce the observation- background biases, show improved performance as the number of assimilated observations increase.
The manuscript provides comprehensive analysis on the impact of massive GNSS-RO observation on NWP system which also gives insight into essential role of GNSS-RO observation. The manuscript is definitely very dense and rich in information based on numerous OSE results. I think the content of the manuscript is very variable to the community and suggest publication after some revisions that might help to clarify the following points:
Other suggestions: