the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 11 Dec 2024
Technical note: An interactive dashboard to facilitate quality control of in-situ atmospheric composition measurements
Abstract. In-situ measurements of trace gases are crucial for monitoring changes in the atmosphere's composition and understanding the underlying processes that drive them. For over three decades, the Global Atmosphere Watch (GAW) programme of the World Meteorological Organization (WMO) has coordinated a network of surface monitoring stations and facilities with the goal of providing high-quality atmospheric composition measurements worldwide. One of the critical challenges towards this goal is the spatially unbalanced availability of high-quality time series, and the lack of near-realtime quality control (QC) procedures that would allow the prompt detection of unreliable data. Here, we describe an interactive dashboard designed for GAW station operators, but which may be of much wider use, that is able to flag anomalous values in near-realtime or historical data. The dashboard combines three distinct algorithms that identify anomalous measurements: (i) an outlier detection based on the Subsequence Local Outlier Factor (Sub-LOF) method, (ii) a comparison with numerical forecasts coupled with a machine learning model, and (iii) a Seasonal Autoregressive Integrated Moving Average (SARIMA) regression model. The application, called GAW-QC, can process measurements of methane (CH4), carbon monoxide (CO), carbon dioxide (CO2), and ozone (O3) at hourly resolution, offering multiple statistical and visual aids to help users to identify problematic data. By enhancing QC capabilities, GAW-QC contributes to the GAW programme's goal of providing reliable atmospheric measurements worldwide.
- Preprint
(1972 KB) - Metadata XML
-
Supplement
(651 KB) - BibTeX
- EndNote
Status: closed
- RC1: 'Comment on egusphere-2024-3556', Anonymous Referee #1, 27 Jan 2025
-
RC2: 'Comment on egusphere-2024-3556', Anonymous Referee #2, 09 Aug 2025
General comment
This technical note by Brugnara et al. illustrates a novel tool for facilitating the quality control (QC) of in-situ atmospheric composition measurements for several WMO/GAW sites. The tool, which is available through an interactive dashboard, is based on three different algorithms for the identification of anomalous measurements.
A general consideration is that the tool gives the possibility either: (i) to analyze already submitted data (e.g., hourly data available at the world data centers, that are already supposed to have been validated by the PIs), and to (ii) submit the user’s own data for a quick check, e.g., before submitting data to the world data centers.
Regarding point (i), I would like the authors to clarify how they envision handling previously validated datasets that the GAW-QC tool subsequently flags as outliers. In such cases, what is the added value of GAW-QC? There is a risk of confusing users if data considered valid by the station PIs are flagged as outliers by GAW-QC.
For point (ii), have the authors considered making the tool integrable into possible existing automatic QC pipelines at WMO/GAW sites under examination? This could allow for seamless, automated usage (e.g., eliminating the need for PIs to manually upload datasets every few months). Such integration could significantly broaden adoption.
The tool is certainly promising and can be widely used within PIs and operators at WMO/GAW stations. The manuscript is overall well written, and I believe it can be published only after the authors better clarify the general use of the tool concerning already validated data, and after addressing the specific comments below.
Specific comments
- Line 25: “critical” can be removed.
- Lines 38–40: this statement may be too general. Near-real-time QC often depends on the capabilities of the station PI, whose responsibility is to check and submit data, typically on an annual basis. Please rephrase to avoid overgeneralization..
- Line 61: “October 24”. This information, as well as that in Fig. 1, can be revised and upgraded in the final version of the manuscript, as many months have passed since October 2024.
- Table 1: does the second column indicate the variable name used in CAMS? If so, please change the column name.
- Line 80: remove “on”.
- Line 85: it is not totally clear to me how the ICOS data are used for validating CAMS data. If the CAMS data are then used to be compared with the observations, how would you treat the GAW stations that are also ICOS sites?
- Line 102: consider presenting the three methods as a numbered list for readability.
- Line 110: very often, to validate or flag some measurements at a station, it is necessary to analyze the behavior of several other compounds (to indicate, e.g., problems in a common sampling head, influence of local pollution, stratospheric intrusion events, ...). As you stated here, the GAW-QC tool does not give the user the possibility of such analysis (which then relies on the PIs’ expertise after a “preliminary” screening with GAW-QC); have the authors thought of providing to the user the possibility of analyzing different variables (already included in GAW-QC) in parallel?
- Line 132: how to deal with missing data records (differently than a timestamp with a missing value, I mean that the data record is totally missing)? Does the input dataset need to be padded, so that all records are present?
- Line 167: “based on expert knowledge”, can you provide more details on this selection?
- Line 219: can you provide the definitions (or examples) of TP, FN, and FP?
- Line 230: by detecting only one outlier per sequence, would then be the responsibility of each PI to accept/select the entire sequence as outliers?
- Line 237: recommended by who? Please specify the source.
- 3 and Fig. 5: do the circles represent the outliers of the distribution? If so, please specify it in the captions.
- 4.2: this section is not totally clear to me: what do you mean by “feature importance”? How is it calculated, and what important information can be retrieved from Fig. 4?
- Lines 319–320: “problems in the measurements”: how can you define/hypothesize that there were problems, given that we must rely on the evaluation of the PIs/station operators for already submitted data?
- Line 367: the format for the user’s input data is specified in the GAW-QC website, but I would add a sentence here on the format to be used.
- Figure 8: colors differ from the web version of GAW-QC; please update the figure and text references accordingly (e.g., “orange shading” at Line 392).
- Line 372: change to “two times or higher than the threshold”.
- Line 383: “flagged data”, do you mean by considering any type of flag?
- Line 385: in the example provided, the histograms suggest CAMS and CAMS+ overestimate measurements. Do the authors have an explanation? I have observed this behavior at multiple stations (marine, high-elevation, etc.) while testing GAW-QC.
- Line 406: what do you mean by this? Aren’t displayed and used data all in UTC by default?
- Line 413: have you confirmed this event by the use of other co-located measurements (e.g., an increase in O3, a decrease in RH), or ancillary variables (e.g., PV)?
- Line 447: this sentence should be rephrased in the final version, as the products may have been upgraded since October 2024.
Citation: https://doi.org/10.5194/egusphere-2024-3556-RC2 -
AC1: 'Reply to referees', Yuri Brugnara, 13 Aug 2025
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-3556/egusphere-2024-3556-AC1-supplement.pdf
Status: closed
-
RC1: 'Comment on egusphere-2024-3556', Anonymous Referee #1, 27 Jan 2025
This manuscript introduced an interactive dashboard tool for detecting anomalous measurements which may help identifying measurement quality issue. This tool is based on three independent algorithms. The algorithms were discussed in the manuscript with application in actual data. The authors correctly pointed out that the anomalous points may or may not be related to measurement problems and the measurement problems should be determined by the instrument scientists or operators. This reviewer believes this tool may help improve measurement quality, but the manuscript did not adequately discuss the effectiveness of this tool. One obvious issue is that not all measurement quality problems are shown as anomalies. The authors should revise the manuscript after considering the comments below.
Major Comments:
- The manuscript primarily focused on discussions about ways to detect anomalous data points. As stated by the authors, the anomalies may not be related to the measurement quality issues. In fact, many unexpected changes in data can be related to swift meteorological condition changes and/or “accidental” emission sources. In general data should not be flagged unless potential specific instrument and/or sampling issues are identified. In this context, it would be helpful for the non-data science readers to see the effectiveness of this tool to identify the measurement problems. This reviewer would like to see the percentage of the identified anomalous data that can be linked to measurement problems and what kind of measurement problems were identified using this tool. This reviewer believes it is an important factor for readers to evaluate the usefulness of the tool if they understand the underlying processes associated with the detected anomalous data.
- It is not widely accepted that the model products can be used to judge measurement problems. This is especially problematic for comparison with ground site observations as the models do not capture local scale meteorology and small emission sources, while both can be reflected in the observations at a given time and location. This may be a more serious problem when dealing with more reactive species.
- The actual measurement problems are often discovered by examine the relationship between variables observed at the same location and time. For example, the stratospheric intrusion case presented in the manuscript should be characterized by higher ozone levels and lower water vapor, in addition to low CO levels. Looking an CO times alone with the model output cannot be considered as convincing… The model can often have a phase shift. It would be much more useful if the tool can take advantage of the simultaneous measurements at a given site.
Specific Comments:
- Table 1.: The authors should explain why and how total column CH4/CO/O3, black carbon, and water vapor would help explain surface observations. This reviewer believes that the readers deserve some physical explanations.
- Line 92: “exceeds 5%”, how sensitive is this choice? Also, why only the “highest sampling height” is selected? Is there often a sharp gradient observed?
- Sub-LOF: the authors should provide more description on the general principle of this algorithm.
- Section 3.2. The authors should discuss the performance of the CAMS model and the difference between the forecast mode and reanalysis mode.
- Line 167: “based on expert knowledge” is not an acceptable justification. The choice of the variables should be explained and justified based on atmospheric sciences.
- Section 3.4.1.: The authors should clearly define true positive, false positive, and false negative. Some examples should be provided to explain these concepts.
- Figure 3: what do the circles represent?
Citation: https://doi.org/10.5194/egusphere-2024-3556-RC1 -
RC2: 'Comment on egusphere-2024-3556', Anonymous Referee #2, 09 Aug 2025
General comment
This technical note by Brugnara et al. illustrates a novel tool for facilitating the quality control (QC) of in-situ atmospheric composition measurements for several WMO/GAW sites. The tool, which is available through an interactive dashboard, is based on three different algorithms for the identification of anomalous measurements.
A general consideration is that the tool gives the possibility either: (i) to analyze already submitted data (e.g., hourly data available at the world data centers, that are already supposed to have been validated by the PIs), and to (ii) submit the user’s own data for a quick check, e.g., before submitting data to the world data centers.
Regarding point (i), I would like the authors to clarify how they envision handling previously validated datasets that the GAW-QC tool subsequently flags as outliers. In such cases, what is the added value of GAW-QC? There is a risk of confusing users if data considered valid by the station PIs are flagged as outliers by GAW-QC.
For point (ii), have the authors considered making the tool integrable into possible existing automatic QC pipelines at WMO/GAW sites under examination? This could allow for seamless, automated usage (e.g., eliminating the need for PIs to manually upload datasets every few months). Such integration could significantly broaden adoption.
The tool is certainly promising and can be widely used within PIs and operators at WMO/GAW stations. The manuscript is overall well written, and I believe it can be published only after the authors better clarify the general use of the tool concerning already validated data, and after addressing the specific comments below.
Specific comments
- Line 25: “critical” can be removed.
- Lines 38–40: this statement may be too general. Near-real-time QC often depends on the capabilities of the station PI, whose responsibility is to check and submit data, typically on an annual basis. Please rephrase to avoid overgeneralization..
- Line 61: “October 24”. This information, as well as that in Fig. 1, can be revised and upgraded in the final version of the manuscript, as many months have passed since October 2024.
- Table 1: does the second column indicate the variable name used in CAMS? If so, please change the column name.
- Line 80: remove “on”.
- Line 85: it is not totally clear to me how the ICOS data are used for validating CAMS data. If the CAMS data are then used to be compared with the observations, how would you treat the GAW stations that are also ICOS sites?
- Line 102: consider presenting the three methods as a numbered list for readability.
- Line 110: very often, to validate or flag some measurements at a station, it is necessary to analyze the behavior of several other compounds (to indicate, e.g., problems in a common sampling head, influence of local pollution, stratospheric intrusion events, ...). As you stated here, the GAW-QC tool does not give the user the possibility of such analysis (which then relies on the PIs’ expertise after a “preliminary” screening with GAW-QC); have the authors thought of providing to the user the possibility of analyzing different variables (already included in GAW-QC) in parallel?
- Line 132: how to deal with missing data records (differently than a timestamp with a missing value, I mean that the data record is totally missing)? Does the input dataset need to be padded, so that all records are present?
- Line 167: “based on expert knowledge”, can you provide more details on this selection?
- Line 219: can you provide the definitions (or examples) of TP, FN, and FP?
- Line 230: by detecting only one outlier per sequence, would then be the responsibility of each PI to accept/select the entire sequence as outliers?
- Line 237: recommended by who? Please specify the source.
- 3 and Fig. 5: do the circles represent the outliers of the distribution? If so, please specify it in the captions.
- 4.2: this section is not totally clear to me: what do you mean by “feature importance”? How is it calculated, and what important information can be retrieved from Fig. 4?
- Lines 319–320: “problems in the measurements”: how can you define/hypothesize that there were problems, given that we must rely on the evaluation of the PIs/station operators for already submitted data?
- Line 367: the format for the user’s input data is specified in the GAW-QC website, but I would add a sentence here on the format to be used.
- Figure 8: colors differ from the web version of GAW-QC; please update the figure and text references accordingly (e.g., “orange shading” at Line 392).
- Line 372: change to “two times or higher than the threshold”.
- Line 383: “flagged data”, do you mean by considering any type of flag?
- Line 385: in the example provided, the histograms suggest CAMS and CAMS+ overestimate measurements. Do the authors have an explanation? I have observed this behavior at multiple stations (marine, high-elevation, etc.) while testing GAW-QC.
- Line 406: what do you mean by this? Aren’t displayed and used data all in UTC by default?
- Line 413: have you confirmed this event by the use of other co-located measurements (e.g., an increase in O3, a decrease in RH), or ancillary variables (e.g., PV)?
- Line 447: this sentence should be rephrased in the final version, as the products may have been upgraded since October 2024.
Citation: https://doi.org/10.5194/egusphere-2024-3556-RC2 -
AC1: 'Reply to referees', Yuri Brugnara, 13 Aug 2025
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-3556/egusphere-2024-3556-AC1-supplement.pdf
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 675 | 57 | 24 | 756 | 36 | 25 | 40 |
- HTML: 675
- PDF: 57
- XML: 24
- Total: 756
- Supplement: 36
- BibTeX: 25
- EndNote: 40
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
This manuscript introduced an interactive dashboard tool for detecting anomalous measurements which may help identifying measurement quality issue. This tool is based on three independent algorithms. The algorithms were discussed in the manuscript with application in actual data. The authors correctly pointed out that the anomalous points may or may not be related to measurement problems and the measurement problems should be determined by the instrument scientists or operators. This reviewer believes this tool may help improve measurement quality, but the manuscript did not adequately discuss the effectiveness of this tool. One obvious issue is that not all measurement quality problems are shown as anomalies. The authors should revise the manuscript after considering the comments below.
Major Comments:
Specific Comments: