the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 15 Aug 2023
Statistical evaluation of methane isotopic signatures determined during near-source measurements
Abstract. Stable carbon isotopic signatures of methane emissions are broadly used for methane source identification, apportionment, and global-scale modelling of methane sources and sinks. Thus, accurate and precise isotopic measurements of methane are crucial for methane studies from the local to global scale. To answer the need for robust and verified measurement methods, we aim at defining the best practice to determine isotopic signatures of methane sources, considering accessibility, practicality, costs, accuracy, and precision. Using Keeling and Miller-Tans methods, we verify the impact of linear fitting methods, averaging approaches, and, for Miller-Tans method, differently defined backgrounds. Verification is carried out for measurement sets using Isotope Ratio Mass Spectrometry and Cavity Ring Down Spectroscopy (CRDS). The use of AirCore for sampling, with subsequent measurements by CRDS, is also examined. Different analytical strategies introduce bias in determining isotopic signatures of methane sources, and the crucial role of rejection criteria is demonstrated. Overall, the most robust results are obtained for non-averaged data using fitting methods, which include uncertainties on x- and y-axis values.
Competing interests: At least one of the (co-)authors is a member of the editorial board of Atmospheric Measurement Techniques.
Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.
Status: closed
-
RC1: 'Comment on egusphere-2023-1490', Anonymous Referee #1, 04 Sep 2023
The authors of the manuscript ‘Statistical evaluation of methane isotopic signatures determined during near-source measurements’ compare the effects from various measurement techniques and analytical methods on δ13C-CH4 source values within a controlled gas release experiment. They intend to give generalized recommendations for the best practice in determing δ13C-CH4 source values during mobile measurements.
In general, because of the controlled conditions, the experimental set-up provides a good basis for studying the effects of near-source measurements. However, I think that the set-up is more suitable for studies that focus on the comparison of measurement methods and the change of environmental conditions such as distance to the source or changes in fluxes.
Comparing different analysis methods (mass conservation method, linear fitting, averaging) is a good idea in principle and could be helpful in establishing best practice guidelines. However, the analysis methods observed here are potentially better suited for computational/modeling studies in which the dominant effects are studied in the context of a multivariate approach.
All in all, I would not recommend this manuscript for publication at the present stage due to the reasons addressed in the general and specific comments below:
general comments:
First of all, the structure of the manuscript is quite poor. For the reader, the content is often confusing and difficult to follow, as basic information, results, discussion and conclusion are wildly mixed in the different manuscript sections and also between the main part of the manuscript and the appendix (see specific comments). Especially because of the numerous analysis parameters to be considered, it is important to create a clear structure, so that the reader does not lose the overview. In this context, Figure 1 is very helpful. It should therefore be used as a "guide" through the manuscript. For example, the analytical methods as well as the results could be presented in the same order as shown in Figure 1 using the same titles, such as: 1. mass conservation method, 2. background subtraction, ... And, similar, for the results: 1. effects from mass conservation, 2. effects from different backgrounds, ...In addition, the various results sections should each contain a corresponding figure showing a comparison of one single parameter. This will contribute to greater clarity and conciseness. A minor point is the clear distinction between the observed δ13CH4 and source δ13CH4 values by using different indices such as ‘o’, ‘s’.
The title is somewhat misleading as it implies a dominance of statistical analysis. However, the manuscript describes more of an application study, investigating numerous parameters based on near-source laboratory measurements and comparing different analytical methods such as mass conservation method or linear fitting method. From the title, I would have expected a manuscript on multivariate statistics to understand the relationships between the numerous parameters and their importance in determining isotopic source values.
Another important criticism is that the novelty of the study is not obvious. The results presented show more or less similar results to previous studies. There are already several publications comparing the different linear fitting methods and mass conservation methods that conclude that the Miller-Tans-Method is the best method when the background value varies, and that the York method is the best fitting method for uncertainties in both, x as well as in y. Some of the results presented here are already self-evident, such as that CRDS uncertainties are higher than those of IRMS measurements due to lower instrument precision.
In my opinion, the requirements for a best-practice recommendation, which the authors have set as their goal, are not met. The given requirements for a generally valid analysis concept are not fulfilled, since the measurements are performed under very specific conditions (synthetic/laboratory) and several problems have not yet been solved, such as the possible temperature effects of the direct measurements or the depletion of CRDS values compared to IRMS values (potential calibration effects).
specific comments:
Abstract:
- line 20: The issues ‘accessibility, practicality and costs’ are not addressed further in the manuscript. I would have expected these topics to be included in the main part as well.
- general: normally, the abstract is a brief summary of the research presented and therefore should also include the main findings of the study, which is what is missing here.
- Introduction:
- line 58: To me, the phrase ‘sampling over five days’ implies a time series. Since you are working with an artificial source that should provide constant values over time, I would rather speak of ‘five consecutive runs’.
- line 62-67: this section rather belongs to the chapter conclusion/outlook
- Controlled release experiment and sampling methodology:
- controlled release set up:
- what is the pressure and volume of the gas cylinders?
- at which location does the release experiment take place?
- is there a control of wind speed and fluxes?
- a more detailed description (besides mentioning of Gardiner et al.) of the controlled release system would be beneficial in this section, as it is the core of the experiment. A graphical scheme would also be helpful.
- Does the decreasing gas pressure of the 12 cylinders have an effect on the isotopic values, since we have sometimes observed a slight change in isotope values with decreasing gas pressure in our standard gas bottles?
- line 68: I would give chapter 2 a more general title like ‘experimental setup’. As the mass conservation method is also an analytical method I would include subchapter 2.4 into chapter 3
- line 80: The use of ‘‰’ to express isotope quantities is obsolete. It is no longer encouraged by IUPAC. According to Brand and Coplen (Assessment of international reference materials for isotope-ratio, 2014), the term ‘milli-Urey’ (mUr) should replace the old ‰ sign. See Werner and Cormier (‘Isotopes-Terminology, Definitions and Properties’, 2022) for current terminology.
- line 83: why was the direct sample from the cylinders not taken both at the beginning and at the end of the experiment to rule out effects such as fluctuating gas pressure?
- line 85: IRMS is not yet explained- please provide a short sentence on instrument specification or a link to the corresponding chapter.
- line 91-105: here the link to Appendix A is missing, where the methods are explained in detail.
- line 110: did I understand this correctly: the Picarro data from the RHUL mobile laboratory was not used for continuous measurement analysis, but only for peak detection for the bag samples? Why not using here an AirCore sampling technique as well?
- Analytical methods of the acquired measurements
- line 160-162: I do not understand that sentence.
- line 162-165: why do you not specify a wide range of different random backgrounds for sensitivity analysis instead of using ‘average’, ‘global’ and ‘random’ backgrounds, some of which are very close to each other?
- line 181: define ‘R’ for the reader who is not familiar with programing languages.
- line 181: what is the mathematical difference between OLS II and MA, since you use the same command in R (lmodel2())?
- line 199: Does it even make sense to test a method other than the one that takes into account the error in x and y such as York and BCES-regression method? Because that is what you would expect from your data which is biased in x as well as in y.
- subchapter 3.5: For a better overview, this subchapter should definitely be placed at the beginning of Chapter 3.
- Results
- line 268-270: A reference for that observation is missing? Table 1?
- line 272: I do not think, that the results are the same. I would rather speak of ‘similar results’.
- line 288-295: this section belongs to the chapter discussion/conclusion
- line 299-300: I did not find information on temperature instability and cavity pressure in Appendix B
- line 301-302: would it be not more appropriate to compare the AirCore values with the CRDS values from Table 2, as both belong to the same method? Or, if you want to use a reference value, why not use those values measured directly from the cylinder because this is more or less the ‘true source’ of δ13C-values?
- line 311 ff: this paragraph is an interpretation and should not be part of the results section
- line 316-321: here, a figure would be helpful to emphasize the results.
- line 321,330 ff: recommendations should be no part of the results section.
- line 339 ff.: this is again an interpretation
- line 351-354: this information is redundant, as it can already be found in subchapter 2.2.
- line 368-369: I do not believe that temperature effects play such a large role in the observed isotope discrepancies. Since direct and indirect measurements are based on completely different conditions (differences in fluxes, flows, mixing effects), I would not have expected unbiased values (deviation from the ‘true’ direct measurements) for the indirectly measured isotope values.
- Discussion
- line 384: The analytical technique used depends primarily on the environmental conditions and requirements, and on the sensitivity required (relative or absolute measurements). For field measurements, it is not always possible to take bag samples and store them for IRMS measurements.
- line 388: The Miller-Tans-method should be used in any case where the background varies (usually in long-term studies). If the background is constant or not measured, the Keeling method can be used. Accordingly, there is not really a question which of the two methods is the better one, because the application depends on measurement conditions.
- line 390: Hoheisel et al. (2019) seem to come to different conclusions: “Especially for natural gas samples, the precise determination and correction of C2H6 is important as in our study C2H6 can bias 13CH4 by up to 3‰ depending on the CH4-to-C2H6 ratio of the sample and the calibration cylinder.” How can you explain this discrepancy?
- line 400-402: there are some more studies for example Takriti et al. (‘Mobile methane measurements: Effects of instrument specifications on data interpretation, reproducibility, and isotopic precision’, 2021) and Hoheisel et al. (2019). So why are you focusing only on the studies mentioned in line 401-402 to compare your results?
- line 415: not only Wehr and Saleska proposed York fitting method, but also many other studies come to the same conclusion such as for example Hoheisel et al. (2019)
- line 420-422: Wehr and Saleska performed model simulations within their study. It is difficult to compare this result with the results of a field test conducted as part of this study.
- line 424-427: what is the conclusion from that statement? Instrument settings have not been discussed before, and introducing another parameter at this point is inappropriate. Therefore, I see no benefit in mentioning this here.
- line 428-429: this is a self-evident fact that applies to all occasions that require sample dilution.
- line 430: I would rather speak of expectable then remarkable (see last comments on results). In general, possible fractionation effects should be checked before starting the actual experiment: e.g. due to different wind speeds, different inflow angles, different temperatures etc. As long as the experimental framework conditions are not clarified, an experiment that investigates numerous parameters is not useful, in my opinion.
- line 436-439: I consider this statement questionable as the experimental conditions are very specific (artificial release of gases to the atmosphere).
- Conclusion
- line 447: I would not speak of statistical methods, but rather of a comparison under different methodological and analytical aspects
Appendix
- Appendix B should appear in a shortened version in the main part of the manuscript (one to two sentences)
- At the end of Appendix B a title (‘Results for bag samples measured on IRMS and CRDS for all examined linear fitting methods’) appears without text following
- Appendix C: A figure would help to clarify the impact of data clustering. I would include this section in the main part of the manuscript.
- for the mobile sampling laboratories, a table with specifications would be beneficial for a better overview
Figures and Tables
- Figure 1: The figure is very clear and helpful to the reader. However, I miss the applied mobile laboratory and the measured components (CH4, δ13CH4) as a note in the graph
- Figure 1: inconsistency: the averaged AirCore samples are presented in ppb- in the main part they are given in nmol/mol
- Figure 3: plotting different CH4 mole fractions based on the size of the data points is absolutely confusing - rather use an average value. A general advice: do not show more than one dependency in one figure
- Figure 3: what does ‘release’ mean? Does this refer to the day of release or to the run number?
- Figure 3: Do you really want to include the data from release 4-5, which is, as you mentioned in the text, biased by calibration errors? Since you also omit the data for averaging I would not show it at all as it confuses the reader.
- Figure B1, C1: these figures are not mentioned anywhere in the text!
- Figure C1: use similar y-axis for better comparison
- Table 3: probably transposed numbers for MA- δ13CH4 Keeling method? (-42.18 instead of -24.18?)
technical corrections:
- please check again for correct super/subscript (i.e., CH4 instead of CH4) throughout the whole manuscript as there are many discrepancies
- line 130 ff: use consistently the term CH4s for source values to distinguish source values from observed values (see formula 1)
- line 31: delete the word ‘measurements’
- line 46: better write ‘equipped with an atmospheric sampling system (AirCore)’ as you have not introduced the AirCore system yet
- line 51: ‘determine δ13C source values’
- line 146: delete commas in that sentence
- line 265-267: ensure that the formulas are displayed correctly: μmol∙mol-1
- line 268: space between treatmen t
- line 339-340: rewrite the sentence as it is not clear (3 times the word corrections!)
-
AC2: 'Reply on RC1', Sara Defratyka, 18 Mar 2024
Answer to Referee 1 Comments on “Statistical evaluation of methane isotopic signatures determined during near-source measurements” by Defratyka et al.,
A: We would like to thank the reviewer for their valuable suggestions that led to an improvement of the paper. In the revised version of the manuscript the structure is improved based on the reviewer’s suggestions and the reviewer’s comments are addressed.
R: The authors of the manuscript ‘Statistical evaluation of methane isotopic signatures determined during near-source measurements’ compare the effects from various measurement techniques and analytical methods on δ13C-CH4 source values within a controlled gas release experiment. They intend to give generalized recommendations for the best practice in determing δ13C-CH4 source values during mobile measurements.
In general, because of the controlled conditions, the experimental set-up provides a good basis for studying the effects of near-source measurements. However, I think that the set-up is more suitable for studies that focus on the comparison of measurement methods and the change of environmental conditions such as distance to the source or changes in fluxes.
A: Thank you for highlighting this aspect of our study. During this controlled release experiment, three measurement methods were used and compared (bag samples measured in the laboratory using IRMS or CRDS and AirCore with CRDS in situ sampling). In the revised version, we better highlight this part of our studies. Also, during the release, methane and ethane were released with varying fluxes and wind speed and direction were measured (added Table A1 in Appendix). Overall, using different measurement methods, we did not observe any significant variation of determined δ13CH4, depending on CH4 flow rate, ratio of C2H6: CH4 or wind speed and direction. We included this observation in the revised version of the manuscript.
R: Comparing different analysis methods (mass conservation method, linear fitting, averaging) is a good idea in principle and could be helpful in establishing best practice guidelines. However, the analysis methods observed here are potentially better suited for computational/modeling studies in which the dominant effects are studied in the context of a multivariate approach.
A: Most of previous studies focused on CO2 or and Monte Carlo simulations (despite Hoheisel et al. focusing on CH4 studies, and using AirCore). Here we focused on real life CH4 dataset, obtained from the controlled release experiment, which simulates real life industrial settings, with a realistic range of CH4 enhancement, number of collected data points used for fitting line and real instrument uncertainty. Thus, we checked the impact of real-life condition for studies of source δ13CH4. Also, we aimed to test all already used fitting approaches from previous studies of mobile CH4 source signature determination and check their potential impact on the final result. Some of these methods were already tested, mostly in context of CO2 respiration. Also, we test here the BCES method which is used for δ13CH4 source of methane but not tested as profoundly as other methods (OLS, ODR, York).
R: All in all, I would not recommend this manuscript for publication at the present stage due to the reasons addressed in the general and specific comments below:
General comments:
R: First of all, the structure of the manuscript is quite poor. For the reader, the content is often confusing and difficult to follow, as basic information, results, discussion and conclusion are wildly mixed in the different manuscript sections and also between the main part of the manuscript and the appendix (see specific comments). Especially because of the numerous analysis parameters to be considered, it is important to create a clear structure, so that the reader does not lose the overview. In this context, Figure 1 is very helpful. It should therefore be used as a "guide" through the manuscript. For example, the analytical methods as well as the results could be presented in the same order as shown in Figure 1 using the same titles, such as: 1. mass conservation method, 2. background subtraction, ... And, similar, for the results: 1. effects from mass conservation, 2. effects from different backgrounds, ...In addition, the various results sections should each contain a corresponding figure showing a comparison of one single parameter. This will contribute to greater clarity and conciseness. A minor point is the clear distinction between the observed δ13CH4 and source δ13CH4 values by using different indices such as ‘o’, ‘s’.
A: Thank you for this important remark. For the revised version, the structure of paper has been improved based on the referee’s comment, leaving the most crucial information and removing less important results. The structure has been corrected based on the suggestion of the reviewer, following the flow chart presented in Figure 1, both for methods and results sections. Section 3.5 has been moved at the beginning of section 3 for better guidance through the paper. The figures and tables have been improved too. Also subscripts ‘obs’ and ’source’ have been added to distinguish observed and source δ13CH4.
R: The title is somewhat misleading as it implies a dominance of statistical analysis. However, the manuscript describes more of an application study, investigating numerous parameters based on near-source laboratory measurements and comparing different analytical methods such as mass conservation method or linear fitting method. From the title, I would have expected a manuscript on multivariate statistics to understand the relationships between the numerous parameters and their importance in determining isotopic source values.
A: Based on the reviewer’s suggestion, we have removed “statistical” from the title, leaving: “Evaluation of methane isotopic signatures determined during near-source measurements”.
R: Another important criticism is that the novelty of the study is not obvious. The results presented show more or less similar results to previous studies. There are already several publications comparing the different linear fitting methods and mass conservation methods that conclude that the Miller-Tans-Method is the best method when the background value varies, and that the York method is the best fitting method for uncertainties in both, x as well as in y. Some of the results presented here are already self-evident, such as that CRDS uncertainties are higher than those of IRMS measurements due to lower instrument precision.
A: Thanks to this comment, we remarked that the novelty of the study was not highlighted enough. In this study, we showed how the lower/ higher precision of instrument translates into a determined source δ13CH4. We focused on CH4 studies, from mobile measurements where number of points and enhancement range are different from CO2 respiratory conditions, which was the main focus of previous studies. Additionally, we included not previously tested BCES, which is another method which includes explicit uncertainty of x and y. It allowed for comparison of two methods with uncertainties (York and BCES), what was not done before. We also checked the impact of used background and averaging for the result. Overall, we tested the impact of the decisions made by a researcher during data analysis on the final δ13CH4 source. Our approach is a validation and implementation of previous results for case of mobile indirect measurements of δ13CH4. The manuscript has been revised accordingly to highlight the importance and novelty of the study: line 64-75: “To derive a more universal analytical approach for near-source studies of δ13CH4 source, isotopic measurement and samples collection were included within a controlled release experiment, which represents real- life emissions from existing methane sources, like natural gas infrastructure or sewage emission. The experiment focused on the validations of methods already applied during mobile, vehicle-based methane measurements (e.g., Hoheisel et al., 2019; Defratyka et al., 2021; Fernandez et al., 2022; Menoud et al., 2022). Samples collected over five consecutive days of the experiment were used to compare Isotope Ratio Mass Spectrometry (IRMS) and Cavity Ring Down Spectroscopy (CRDS) measurement techniques (Sect. 2.2), while CRDS instrument was used both for in-situ sampling using AirCore tool, and for remote bag analysis after the sample collection. Comparing these three types of measurement technique, we provide insight into results from high precision (IRMS) and low precision (CRDS) instruments and impact of increasing data frequency for low precision instrument (CRDS with AirCore). Moreover, the studies were focused on a comprehensive inter-comparison of Keeling and Miller-Tans methods (Sect. 3.1), the impact of chosen backgrounds for Miller-Tans method (Sect. 3.2) and impact of averaging clusters (Sect. 3.3). Finally, data were re-analysed using different linear fitting methods (Sect. 3.4).” and: 436-444: “The novelty of the study is the comprehensive inter-comparison between indirect studies of δ13CH4 using (i) bag sampling measured afterwards both by IRMS and CRDS, (ii) in-situ CRDS with an AirCore storage tool under controlled release conditions. Also, we focused on intercomparison of different analytical strategies already used in indirect studies of δ13CH4 source, which was not done before on such a broad scale. We tested aspects which were not detailed in previous studies, like background subtraction, data averaging and BCES linear fitting. To achieve it, we focused on data from controlled release experiment, which is a representation of real-life point methane sources, like for example leaks in natural gas infrastructure. This approach allows for validation and implementation of conclusions from studies mostly focused on synthetic or CO2 data (Pataki et al., 2003; Miller and Tans, 2003; Zobitz et al., 2006; Wehr and Saleska, 2017).”
R: In my opinion, the requirements for a best-practice recommendation, which the authors have set as their goal, are not met. The given requirements for a generally valid analysis concept are not fulfilled, since the measurements are performed under very specific conditions (synthetic/laboratory) and several problems have not yet been solved, such as the possible temperature effects of the direct measurements or the depletion of CRDS values compared to IRMS values (potential calibration effects).
A: Overall, controlled release experiment is used as a representation of existing methane point sources in a range of typical industrial settings, as described in Gardiner et al. (2017). As studies are focused on mobile measurements, presented results are applicable to all mobile measurements using bag sampling or in situ sampling with AirCore, which are common techniques for δ13CH4 source determination. In the revised version of the manuscript, we have changed from the general best practice recommendations to recommendation for case studies of mobile measurements of CH4.
Regarding potential discrepancies between CRDS and IRMS results, numerous testes of CRDS instruments were done and details can be found in (Defratyka, chapter 2, 2021). No significant impact of temperature or pressure for CRDS results was observed. δ13CH4 of used calibration cylinders was assigned at RHUL, ensuring measurements on the same scale. In the revised manuscript, the following text has been added: lines 115-124 “Due to logistic restrictions, the cylinder with calibration or known working gas could not be used during the controlled release experiment. However, an additional working gas was measured in the laboratory the day before and the day after the experiment and no significant drift was observed. During other independent measurement campaigns, an additional working gas was measured on the CRDS used here, before and/or after randomly chosen surveys (Defratyka, 2021, chapter 2). Observed time drift was negligible and working gas measurements inside the vehicle did not improve instrument precision. Thus, 3-point calibration done before controlled release experiment gives robust results. Additionally, the instrument was tested in laboratory conditions to verify possible impact of external pressure and temperature for CRDS instrument. No significant dependence on the ambient pressure and temperature both for CH4 mole fraction and δ13CH4 was observed for CRDS instrument used during this study (Defratyka, 2021, chapter 2).”
specific comments:
Abstract:
- line 20: The issues ‘accessibility, practicality and costs’ are not addressed further in the manuscript. I would have expected these topics to be included in the main part as well.
A: In the revised version of the manuscript, “accessibility, practicality and costs” have been removed from the abstract.
- general: normally, the abstract is a brief summary of the research presented and therefore should also include the main findings of the study, which is what is missing here.
A: An additional part with main finding of the study have been included in the abstract in the revised version: “Due to high precision and accuracy of IRMS instruments, the chosen analytical strategy does not significantly affect IRMS results. However more precautions must be made for analysis of CRDS AirCore samples. Fitting methods with forced symmetry like Major Axis or Bivariate Correlated Errors and Intrinsic Scatter (BCES) Orthogonal introduce significant biases in the determined isotopic signatures of methane sources. The most reliable results are obtained for non-averaged data using fitting methods, which include uncertainties on x- and y-axis values, like York fitting or BCES (Y|X). Ordinary Least Squares method provides sufficiently robust results and, for simplicity, can be used to determine δ13CH4 in near-source conditions.”
Introduction:
- line 58: To me, the phrase ‘sampling over five days’ implies a time series. Since you are working with an artificial source that should provide constant values over time, I would rather speak of ‘five consecutive runs’.
A: the phrase has been corrected in the revised version.
- line 62-67: this section rather belongs to the chapter conclusion/outlook
A: This part has been removed from revised version.
Controlled release experiment and sampling methodology:
- controlled release set up:
- what is the pressure and volume of the gas cylinders?
- at which location does the release experiment take place?
- is there a control of wind speed and fluxes?
- a more detailed description (besides mentioning of Gardiner et al.) of the controlled release system would be beneficial in this section, as it is the core of the experiment. A graphical scheme would also be helpful.
- Does the decreasing gas pressure of the 12 cylinders have an effect on the isotopic values, since we have sometimes observed a slight change in isotope values with decreasing gas pressure in our standard gas bottles?
A: In revised version of the manuscript, description of controlled experiment set up has been more detailed, to answer all Referee’s questions: lines 79-93: “The experiment lasted over 5 days in September 2019 at Bedford Aerodrome, UK. Pure methane was released from a manifolded multi-cylinder pack, of eleven cylinders containing 999.6 ± 10.0 mmol mol-1, with initial pressure about 200 bars. The impurities in cylinders came from ethane (48 ± 10 µmol mol-1) and propane (0.149 ± 0.30 µmol mol-1). All 11 cylinders were filled at the same time from the same CH4 source, ensuring δ13CH4 source remained stable over the entire measurement period. The methane release rate varied between releases, up to 70 L min-1. During the release, CH4 was mixed with ethane (C2H6) in a varying ratio, giving C2H6:CH4 ratios from 0.00 to 0.07. The purity of the C2H6 was 999.9 ± 10.0 mmol mol-1, with impurities mostly from methane (2.27 ± 0.46 µmol mol-1) and propane (7.5 ± 1.5 µmol mol-1). To achieve the required CH4 emission rate, CH4 was released simultaneously from all 11 cylinders, using the transportable flow control system (details in Gardiner et al., 2017)). Briefly, the flow control system is designed and configured for the creation of ‘real-life’ gaseous emission scenarios, in a range of industrial settings. It allows for the validation of methods for emission monitoring applied during typical field conditions. The control system is built on six mass flow controllers (MFC), (Brooks Instrument, Hatfield, PA, USA). Four primary MFCs provide four independent emission sources, while two secondary MFCs allow for the introduction of purge and interferant gases into the primary flow. The flow control system is computer operated, allowing for the implementation of pre-written operational programs and the post-test analysis (Gardiner et al., 2017).”
Regarding the question of decreasing gas pressure of the 12 cylinders, we do not expect any effect from the decrease of the pressure on δ13CH4. For clarification, in the first version of manuscript we wrote 12 cylinders instead of 11 cylinders. First, the gas was released from 11 cylinders in the batch simultaneously, using controlling station. Second, to verify the possibility of δ13CH4 fractionation during the experiment, last summer during an additional control release experiment, three samples were taken directly from cylinder batch, similar to way it was done during our controlled release experiment. First sample was taken when cylinder pack was first used. Second sample was taken immediately after we had been withdrawing >50 litres/min from it for 1 hour. Cylinders slightly cool, pressure regulator iced. And third sample was taken on the last day of tests but before any tests had started for the day. No statistically significant differences were observed. This reject the hypothesis of decreasing gas pressure of the 11 cylinders having an effect on the isotopic values and also the hypothesis about the potential impact of temperature for fractionation between direct and indirect sampling, which was initially mentioned in the first version of the manuscript and has been removed for revised version.
- line 68: I would give chapter 2 a more general title like ‘experimental setup’. As the mass conservation method is also an analytical method I would include subchapter 2.4 into chapter 3
A: The proposed changes have been implemented in the revised version.
- line 80: The use of ‘‰’ to express isotope quantities is obsolete. It is no longer encouraged by IUPAC. According to Brand and Coplen (Assessment of international reference materials for isotope-ratio, 2014), the term ‘milli-Urey’ (mUr) should replace the old ‰ sign. See Werner and Cormier (‘Isotopes-Terminology, Definitions and Properties’, 2022) for current terminology.
A: The “‰” is still widely used in the atmospheric community to express the isotopic ratio of δ13CH4. For example, results of flask samples presented at NOAA website are also expressed in ‰ unit. Also, recent papers focusing on continuous measurements of δ13CH4 use ‰ unit (Hoheisel and Schmidt, 2023; Maisch et al., 2023). Moreover, broadly used international guidelines also use ‰ (Coplen, 2011, cited <1300 times; Brand et al., 2014, cited <500 times). Also, CCQM working Group on Isotope Ratios in their strategy for 2021-2030 uses ‰ (CCQM, 2021). Additionally, readers are much more familiar with ‰ unit than with new proposed milli-Urey unit. Thus, respectfully, we will stay with ‰ unit, as in our opinion it is more suitable for our study.
- line 83: why was the direct sample from the cylinders not taken both at the beginning and at the end of the experiment to rule out effects such as fluctuating gas pressure?
A: We did not expect observing changes of isotopic signature during release thus we took a direct sample only at the end of the experiment. However, as obtained result posed question about potential impact of gas pressure in cylinder and temperature of released gas for δ13CH4, during another controlled release experiment (summer 2023), we took three samples directly from the multi-cylinder pack, as explained above. No statistically significant differences were observed. This rejects the hypothesis of decreasing gas pressure of the 11 cylinders influencing the isotopic values during our controlled release experiment.
- line 85: IRMS is not yet explained- please provide a short sentence on instrument specification or a link to the corresponding chapter.
A: In revised version of the manuscript, description of bag samples measured on IRMS (Sect. 2.2.1) has been moved before explication of direct sampling (2.2.4), and explication of the instrument is in section 2.2.1.
- line 91-105: here the link to Appendix A is missing, where the methods are explained in detail.
A: A link has been added in revised version.
- line 110: did I understand this correctly: the Picarro data from the RHUL mobile laboratory was not used for continuous measurement analysis, but only for peak detection for the bag samples? Why not using here an AirCore sampling technique as well?
A: Yes, that is correct. RHUL mobile laboratory uses Picarro model G2301 which does not measure δ13CH4. The information has been added in the manuscript, and table with used instrument specification has been added in Appendix: lines 103-105 “This vehicle was equipped with a Picarro CRDS G2301 analyser, capable to measure mole fraction of CO2, CH4 and H2O, a Los Gatos Research Ultraportable Methane Ethane Analyzer (LGR UMEA) and a manually operated diaphragm pump for air sample bag filling (Table A2).”
Analytical methods of the acquired measurements
- line 160-162: I do not understand that sentence.
A: The sentence has been revised for clarification: : “For random I background, CH4 bckg mole fraction is defined as an average global CH4 mole fraction observed in September 2019, equals to 1.8707 ± 0.0011 µmol mol-1 (NOAA/ESRL), while δ13CH4 bckg is defined using value from Brownlow et al., as -47.2 ± 0.2 ‰ (2017), to verify if slight difference in background can translate into significant differences in determined δ13CH4 source.”
- line 162-165: why do you not specify a wide range of different random backgrounds for sensitivity analysis instead of using ‘average’, ‘global’ and ‘random’ backgrounds, some of which are very close to each other?
A: This part has been improved in revised manuscript: lines 211-219 “Next, to verify the sensitivity of Miller-Tans method for a differently defined background, calculations for two randomly chosen backgrounds with lower CH4 bckg mole fraction and δ13CH4 bckg than during the experiment were conducted: “random I” and “random II” background. For random I background, CH4 bckg mole fraction is defined as an average global CH4 mole fraction observed in September 2019, equals to 1.8707 ± 0.0011 µmol mol-1 (NOAA/ESRL), while δ13CH4 bckg is defined using value from Brownlow et al., as -47.2 ± 0.2 ‰ (2017), to verify if slight difference in background can translate into significant differences in determined δ13CH4 source. For random II background, the CH4 bckg mole fraction is set up the same as the random I background, but the δ13CH4 bckg was randomly set to -42.7 ± 0.2 ‰ to significantly differ from other δ13CH4 bckg backgrounds to better test the sensitivity of Miller-Tans method to subtracted background.”
- line 181: define ‘R’ for the reader who is not familiar with programing languages.
A: It has been rewritten as: line 262 “Most of the tested fitting are calculated using built in packages and functions in R programming language…”
- line 181: what is the mathematical difference between OLS II and MA, since you use the same command in R (lmodel2())?
A: Function lmodel2() allows to calculate slope and intercept of linear fitting using different fitting methods: OLS, MA, SMA (standard major axis) and RMA (ranged major axis). Thus, OLS II should give the same results as lm() function in R which calculates slope and intercept only using OLS method. As expected, these two functions give the same results for OLS fitting, although in some cases differences with uncertainties were observed. In revised version of the manuscript, we have removed all OLS II calculations for clarification of the manuscript, as adding this fitting does not bring any additional strategy comparison and it is rather internal R language packages comparison.
- line 199: Does it even make sense to test a method other than the one that takes into account the error in x and y such as York and BCES-regression method? Because that is what you would expect from your data which is biased in x as well as in y.
A: We decided to include also methods which do not take into account errors in x and y (OLS and MA) as they were used in previous studies for determination of source signature (e.g. Defratyka, 2021; Menoud et al., 2022; Takriti et al., 2023). Additionally, even if methods like York and BCES exist, OLS is still the most used fitting method. Thus, including OLS and MA allows for testing potential bias implemented by fitting methods without errors.
- subchapter 3.5: For a better overview, this subchapter should definitely be placed at the beginning of Chapter 3.
A: The section 3.5 has been moved to the beginning of section 3.
Results
- line 268-270: A reference for that observation is missing? Table 1?
A: The reference to Figure 3 will be added in revised version.
- line 272: I do not think, that the results are the same. I would rather speak of ‘similar results’.
A: “the same results” has been replaced by “similar results”.
- line 288-295: this section belongs to the chapter discussion/conclusion
A: This part has been removed from the manuscript as it does not belong to results and it is repetition of already existing discussion.
- line 299-300: I did not find information on temperature instability and cavity pressure in Appendix B
A: The information about temperature and cavity pressure instability was included in Appendix A instead of Appendix B (lines 593-596). The correct appendix has been mentioned in the revised version.
- line 301-302: would it be not more appropriate to compare the AirCore values with the CRDS values from Table 2, as both belong to the same method? Or, if you want to use a reference value, why not use those values measured directly from the cylinder because this is more or less the ‘true source’ of δ13C-values?
A: The three measurement method have been compared in the same section (Sect. 2.2 for methods and 4.1 for results), following the order presented in flow chart on Figure 1, for easier following by the reader. For better clarification, we defined “reference analytical strategy: lines 312-314: “For comparison between different set up, we are using results from Miller-Tans method with individual background subtracted and York fitting method, and treatment 1 averaging for bag samples or raw data for AirCore samples as the reference analytical strategy.”, which is used through the manuscript. We decided to use IRMS results instead of direct results as a reference value for indirect results, as we observe significant discrepancy between direct sampling and IRMS results. Thus, comparison only between indirect methods using IRMS reference value allows for better emphasis of observed biases introduced by different analytical strategies.
- line 311 ff: this paragraph is an interpretation and should not be part of the results section
A: This part has been removed from results.
- line 316-321: here, a figure would be helpful to emphasize the results.
A: This paragraph has been moved to another section, focused on averaging, to keep the order from flow chart in Figure 1. Figure 4 has been added for visualisation of the results.
- line 321,330 ff: recommendations should be no part of the results section.
A: This part has been removed from results.
- line 339 ff.: this is again an interpretation
A: This part has been removed from results.
- line 351-354: this information is redundant, as it can already be found in subchapter 2.2.
A: This part has been removed from results.
- line 368-369: I do not believe that temperature effects play such a large role in the observed isotope discrepancies. Since direct and indirect measurements are based on completely different conditions (differences in fluxes, flows, mixing effects), I would not have expected unbiased values (deviation from the ‘true’ direct measurements) for the indirectly measured isotope values.
A: We have tried to reword the main text in reply to the reviewer’s concern: lines 522-530 “We observed about a 1.1 ‰ discrepancy between indirectly (IRMS results) and directly determined δ13CH4 source. The observed discrepancy could be caused by different conditions in which direct and indirect samples are collected. For indirect studies, the gas was released over 45 minutes from the cylinder at high rates (up to 70 l min-1). For direct sampling, gas was released from one cylinder to another in less than two minutes. During the controlled release experiment we did not observe significant differences between δ13CH4 source indirectly determined during different releases when CH4 flow rate, wind speed, or wind direction varied (Tab. A1). These physical phenomena alone only impact the dispersion characteristic of CH4 from the source to the sampling location, without any plausible mechanism for isotopic fractionation within the boundary layer on these short timescales. Further studies on possible isotopic fractionation during gas release are planned in the future to verify observed discrepancy.”
Discussion
- line 384: The analytical technique used depends primarily on the environmental conditions and requirements, and on the sensitivity required (relative or absolute measurements). For field measurements, it is not always possible to take bag samples and store them for IRMS measurements.
A: Here, we wanted to highlight that if bags are collected, they should be measured on IRMS instead of CRDS and that using AirCore with CRDS gives better results than measuring bags on CRDS in the laboratory. We have rewritten the sentence to clarify the conclusion: lines 451-453 “Due to the bigger sensitivity to applied analytical strategy and higher uncertainty, comparing to IRMS results, we do not recommend measuring bag samples using CRDS. To use CRDS instrument for determination of δ13CH4S, AirCore tool should be implemented for in-situ CRDS sampling, as this measurement techniques gives more robust results.”
- line 388: The Miller-Tans-method should be used in any case where the background varies (usually in long-term studies). If the background is constant or not measured, the Keeling method can be used. Accordingly, there is not really a question which of the two methods is the better one, because the application depends on measurement conditions.
A: There are not clear thresholds to define when one method is preferred over another; there is a continuum of conditions over which either method could be applied. Our mobile measurements describe such complexity where we observe also a short term change of background CH4 mole fraction. In the morning, after night-time accumulation, CH4 mole fraction can be significantly higher than the daily average and decrease with time, especially during winter months. Moreover, δ13CH4 of sources are compared between each other in different locations and to observe possible changes over time (e.g., seasons, years). Thus, Miller-Tans method is more suitable for mobile measurements even though there are short-term studies of one individual source. As some groups use or Miller-Tans or Keeling method, we compared results of both of them to see if any method implement bias and can affect comparison of results from different groups.
- line 390: Hoheisel et al. (2019) seem to come to different conclusions: “Especially for natural gas samples, the precise determination and correction of C2H6 is important as in our study C2H6 can bias 13CH4 by up to 3‰ depending on the CH4-to-C2H6 ratio of the sample and the calibration cylinder.” How can you explain this discrepancy?
A: During the control release experiment, we were using exactly the same instrument as Assan et al. (2017) used to define and test ethane correction. Tests conducted by Assan et al. (2017), were repeated and results are in good agreement with previous studies (Defratyka, chapter 2 and 3, 2021). The same methods were used by Hoheisel et al. (2019) giving similar results. Overall, applying ethane correction, the δ13CH4 is shifted toward more depleted values as expected from previous studies (Rella et al., 2015; Assan et al., 2017; Hoheisel et al., 2019). However, the previous studies did not compare obtained CRDS results with IRMS studies or with direct sampling, what we do here. Potentially, it opens room for the discussion if ethane correction should be used or not. We have mentioned it in revised version of the manuscript: lines 465-469 “According to our knowledge, this is the first study, when CRDS AirCore results are directly compared to δ13CH4 source determined from bag samples measured on IRMS and direct source sampling, and providing evidence to exclude an C2H6 correction for measuring δ13CH4 in these types of samples. Previous studies implementing an C2H6 correction focused only on CRDS AirCore studies without the comparison with independent C2H6-interference free measurements. (Rella et al., 2015; Assan et al., 2017; Lopez et al., 2017; Hoheisel et al., 2019).”
- line 400-402: there are some more studies for example Takriti et al. (‘Mobile methane measurements: Effects of instrument specifications on data interpretation, reproducibility, and isotopic precision’, 2021) and Hoheisel et al. (2019). So why are you focusing only on the studies mentioned in line 401-402 to compare your results?
A: In the discussion comparison with Hoheisel et al. (2019) has been added: 489-501 “Hoheisel et al. (2019) focused on comparison of Keeling and Miller-Tans methods, using OLS and York fitting, for CH4 obs AirCore and synthetic data. Regarding comparison of measurement techniques, they obtained identical results for Keeling and Miller-Tans methods using York fitting. Using OLS fitting, they observed differences from -2 ‰ to 2 ‰ for individual AirCore samples between Keeling and Miller-Tans methods. Hoheisel et al. (2019) showed that in the case of AirCore studies, results of York fitting lie between results from OLS using Keeling or Miller-Tans method for 90% of measurements. In their study, in the case of synthetic data, results from York and OLS are nearly the same, having larger differences to true, modelled value. The observed discrepancy between fitted and true value achieved <0.2 ‰. These results are consistent with our study, where we did not observe significant differences between York and OLS fitting and bias between indirect and direct results achieved 0.3 ‰ for AirCore samples. Additionally, Hoheisel et al. (2019) checked influence of averaging time up to 1 minute. Using synthetic data, they demonstrated no significant differences between raw and 15 s averaged. Also, they observed the improvement of precision of the measurement during averaging over 1 minute, which did not improve δ13CH4 source signature determination. This is in opposition to our results from mobile measurements, where we observe the varying bias introduced by data averaging, which worsen determination on final δ13CH4 source. ”
We decided to not include comparison with Takriti et al., (2021), as their study focus on instrument performance tests, especially regarding response time. Some of discussed issues were previously tested in details by Hoheisel et al. (2019). Additionally, used in our studies CRDS had higher than default flow rate (100 ml/min versus typical 25 ml/min) to increase response time. Moreover, for in-situ sampling we used AirCore tool. Thus, increased flow rate and using AirCore tool makes difficult comparing between our set up and setup presented by Takriti et al., (2021).
- line 415: not only Wehr and Saleska proposed York fitting method, but also many other studies come to the same conclusion such as for example Hoheisel et al. (2019)
A: We have also included conclusion of Hoheisel et al. (2019) in the revised version (lines 479-491).
- line 420-422: Wehr and Saleska performed model simulations within their study. It is difficult to compare this result with the results of a field test conducted as part of this study.
A: Based on our understanding of the work of Wehr and Saleska, they created synthetic data as a representation of real-life dataset and using Monte Carlo simulation they compare results for York, OLS and GMR fitting. Here, we use real-life set of data to observe impact of different analytical strategies for determined δ13CH4. Thus, our case study are validation and implementation of results from synthetic data studies to real-life measurements.
- line 424-427: what is the conclusion from that statement? Instrument settings have not been discussed before, and introducing another parameter at this point is inappropriate. Therefore, I see no benefit in mentioning this here.
A: Thank you for highlighting this. We have removed this part from the revised manuscript.
- line 428-429: this is a self-evident fact that applies to all occasions that require sample dilution.
A: This part has been removed from revised version.
- line 430: I would rather speak of expectable then remarkable (see last comments on results). In general, possible fractionation effects should be checked before starting the actual experiment: e.g. due to different wind speeds, different inflow angles, different temperatures etc. As long as the experimental framework conditions are not clarified, an experiment that investigates numerous parameters is not useful, in my opinion.
A: Isotopic fractionation at the point of release is the only plausible way that we understand could affect isotopic composition during transport from the primary source to sample location. In the revised version we have added paragraph about observed discrepancies: lines 522-530 “We observed about a 1.1 ‰ discrepancy between indirectly (IRMS results) and directly determined δ13CH4 source. The observed discrepancy could be caused by different conditions in which direct and indirect samples are collected. For indirect studies, the gas was released over 45 minutes from the cylinder at high rates (up to 70 l min-1). For direct sampling, gas was released from one cylinder to another in less than two minutes. During the controlled release experiment we did not observe significant differences between δ13CH4 source indirectly determined during different releases when CH4 flow rate, wind speed, or wind direction varied (Tab. A1). These physical phenomena alone only impact the dispersion characteristic of CH4 from the source to the sampling location, without any plausible mechanism for isotopic fractionation within the boundary layer on these short timescales. Further studies on possible isotopic fractionation during gas release are planned in the future to verify observed discrepancy.”
- line 436-439: I consider this statement questionable as the experimental conditions are very specific (artificial release of gases to the atmosphere).
A: The control release experiment is a good representation of industrial settings and real life the conditions when mobile measurements of methane sources are made. Thus, we conduct the analysis of available analytical methods implied to real life data, when we can observe true impact of instrument (frequency, precision), methane range and meteorological conditions. Additionally, we have rewritten the manuscript to “a case study” instead of “general recommendation”.
Conclusion
- line 447: I would not speak of statistical methods, but rather of a comparison under different methodological and analytical aspects
A: The sentence has been rewritten.
Appendix
- Appendix B should appear in a shortened version in the main part of the manuscript (one to two sentences)
A: In the revised version, the appendix B has been shortly explained in main part of the manuscript.
- At the end of Appendix B a title (‘Results for bag samples measured on IRMS and CRDS for all examined linear fitting methods’) appears without text following
A: This part of Appendix B contains only figures and tables which were added at the end of the manuscript. In the revised manuscript, Appendix tables and figures are moved into appendix part of the manuscript.
- Appendix A: A figure would help to clarify the impact of data clustering. I would include this section in the main part of the manuscript.
A: The main findings of clustering impact have been added in the main text and figure 4 has been added.
- for the mobile sampling laboratories, a table with specifications would be beneficial for a better overview
A: A table A2 with specification of mobile laboratories and used instruments has been added in Appendix A.
Figures and Tables
- Figure 1: The figure is very clear and helpful to the reader. However, I miss the applied mobile laboratory and the measured components (CH4, δ13CH4) as a note in the graph
- Figure 1: inconsistency: the averaged AirCore samples are presented in ppb- in the main part they are given in nmol/mol
A: Figure 1 is improved based on reviewer’s comment.
- Figure 3: plotting different CH4 mole fractions based on the size of the data points is absolutely confusing - rather use an average value. A general advice: do not show more than one dependency in one figure
A: Figure has improved in revised manuscript.
- Figure 3: what does ‘release’ mean? Does this refer to the day of release or to the run number?
A: Here the releases are numbered in following order X.Y, where X stands for day and Y for the release made during individual day (e.g. 1.2 stands for the second release of the first day of the experiment).
- Figure 3: Do you really want to include the data from release 4-5, which is, as you mentioned in the text, biased by calibration errors? Since you also omit the data for averaging I would not show it at all as it confuses the reader.
A: In the revised version, the observed discrepancy is mentioned but data from day 4-5 are still included, as observed values are in agreement within uncertainty and there passed through rejection criteria. We will add in revised version: lines 514-519 “We observed that individual AirCore values for samples collected on days 4 and 5 of the controlled release experiment are more depleted than samples collected in days 2 and 3 (Fig. 2), but δ13CH4 source results are still in agreement within uncertainty. It is possible, that an unnoticed problem occurred with the instrument calibration or encountered mobile set-up leaks during those days. Based on this, we recommend measuring the calibration gases on each measurement day, both before and after the fieldwork.
- Figure B1, C1: these figures are not mentioned anywhere in the text!
A: These figures have been removed from the revised manuscript.
- Figure C1: use similar y-axis for better comparison
A: This figure has been removed from the revised manuscript.
- Table 3: probably transposed numbers for MA- δ13CH4 Keeling method? (-42.18 instead of -24.18?)
A: This value was double checked and there was no transposed number here. As explained in the text MA fitting introduce significant bias and value -24.18 ‰ is what we received from the calculation.
A: All proposed corrections for Figures and Tables have been implemented in the revised version.
technical corrections:
- please check again for correct super/subscript (i.e., CH4 instead of CH4) throughout the whole manuscript as there are many discrepancies
- line 130 ff: use consistently the term CH4s for source values to distinguish source values from observed values (see formula 1)
- line 31: delete the word ‘measurements’
- line 46: better write ‘equipped with an atmospheric sampling system (AirCore)’ as you have not introduced the AirCore system yet
- line 51: ‘determine δ13C source values’
- line 146: delete commas in that sentence
- line 265-267: ensure that the formulas are displayed correctly: μmol∙mol-1
- line 268: space between treatmen t
- line 339-340: rewrite the sentence as it is not clear (3 times the word corrections!)
A: All proposed technical corrections have been implemented in the revised version.
Citation: https://doi.org/10.5194/egusphere-2023-1490-RC1
References
Assan, S., Baudic, A., Guemri, A., Ciais, P., Gros, V., and Vogel, F. R.: Characterization of interferences to in situ observations of d13CH4 and C2H6 when using a cavity ring-down spectrometer at industrial sites, Atmospheric Meas. Tech., 10, 2077–2091, https://doi.org/10.5194/amt-10-2077-2017, 2017.
Brand, W. A., Coplen, T. B., Vogl, J., Rosner, M., and Prohaska, T.: Assessment of international reference materials for isotope-ratio analysis (IUPAC Technical Report), Gruyter, 86, 425–467, https://doi.org/10.1515/pac-2013-1023, 2014.
Brownlow, R., Lowry, D., Fisher, R. E., France, J. L., Lanoisellé, M., White, B., Wooster, M. J., Zhang, T., and Nisbet, E. G.: Isotopic Ratios of Tropical Methane Emissions by Atmospheric Measurement: Tropical Methane δ13C Source Signatures, Glob. Biogeochem. Cycles, 31, 1408–1419, https://doi.org/10.1002/2017GB005689, 2017.
CCQM: Consultative Committee for Amount of substance; Metrology in Chemistry and Biology CCQM Working Group on Isotope Ratios (IRWG) Strategy for Rolling Programme Development (2021-2030), https://www.bipm.org/documents/20126/57465585/CCQM-IRWG+Strategy+document+2021-2030.pdf/41d93edc-c543-8ed4-883b-26e97ac93867, 2021.
Coplen, T. B.: Guidelines and recommended terms for expression of stable-isotope-ratio and gas-ratio measurement results: Guidelines and recommended terms for expressing stable isotope results, Rapid Commun. Mass Spectrom., 25, 2538–2560, https://doi.org/10.1002/rcm.5129, 2011.
Defratyka, S.: Characterizing methane (CH4) emissions in urban environments (Paris), Paris-Saclay, Gif sur Yvette, 185 pp., 2021.
NOAA/ESRL: https://gml.noaa.gov/ccgg/trends_ch4/.
Gardiner, T., Helmore, J., Innocenti, F., and Robinson, R.: Field Validation of Remote Sensing Methane Emission Measurements, Remote Sens., 9, 956, https://doi.org/10.3390/rs9090956, 2017.
Hoheisel, A. and Schmidt, M.: Six years of continuous carbon isotope composition measurements of methane in Heidelberg (Germany) − a study of source contributions and comparison to emission inventories, 2023.
Hoheisel, A., Yeman, C., Dinger, F., Eckhardt, H., and Schmidt, M.: An improved method for mobile characterisation of D13CH4 source signatures and its application in Germany, Atmospheric Meas. Tech., 12, 1123–1139, https://doi.org/10.5194/amt-12-1123-2019, 2019.
Lopez, M., Sherwood, O. A., Dlugokencky, E. J., Kessler, R., Giroux, L., and Worthy, D. E. J.: Isotopic signatures of anthropogenic CH4 sources in Alberta, Canada, Atmos. Environ., 164, 280–288, https://doi.org/10.1016/j.atmosenv.2017.06.021, 2017.
Maisch, C. A. W., Fisher, R. E., France, J. L., Lowry, D., Lanoisellé, M., Bell, T. G., Forster, G., Manning, A. J., Michel, S. E., Ramsden, A. E., Yang, M., and Nisbet, E. G.: Methane Source Attribution in the UK Using Multi‐Year Records of CH4 and δ13C, J. Geophys. Res., 2023.
Menoud, M., van der Veen, C., Lowry, D., Fernandez, J. M., Bakkaloglu, S., France, J. L., Fisher, R. E., Maazallahi, H., Stanisavljević, M., Nęcki, J., Vinkovic, K., Łakomiec, P., Rinne, J., Korbeń, P., Schmidt, M., Defratyka, S., Yver-Kwok, C., Andersen, T., Chen, H., and Röckmann, T.: New contributions of measurements in Europe to the global inventory of the stable isotopic composition of methane, Earth Syst. Sci. Data, 14, 4365–4386, https://doi.org/10.5194/essd-14-4365-2022, 2022.
Rella, C. W., Hoffnagle, J., He, Y., and Tajima, S.: Local- and regional-scale measurements of CH4, d13CH4, and C2H6; in the Uintah Basin using a mobile stable isotope analyzer, Atmospheric Meas. Tech., 8, 4539–4559, https://doi.org/10.5194/amt-8-4539-2015, 2015.
Takriti, M., Wynn, P. M., Elias, D. M. O., Ward, S. E., Oakley, S., and McNamra, N. P.: Mobile methane measurements: Effects of instrument specifications on data interpretation, reproducibility, and isotopic precision, Atmos. Environ., 246, https://doi.org/doi.org/10.1016/j.atmosenv.2020.118067, 2021.
Takriti, M., Ward, S. E., Wynn, P. M., and McNamra, N. P.: Isotopic characterisation and mobile detection of methane emissions in a heterogeneous UK landscape, Atmos. Environ., 305, https://doi.org/10.1016/j.atmosenv.2023.119774, 2023.
Citation: https://doi.org/10.5194/egusphere-2023-1490-AC2 -
AC3: 'Reply on RC1', Sara Defratyka, 18 Mar 2024
Publisher’s note: this comment is a copy of AC2 and its content was therefore removed.
Citation: https://doi.org/10.5194/egusphere-2023-1490-AC3 -
AC4: 'Reply on RC1', Sara Defratyka, 18 Mar 2024
Publisher’s note: this comment is a copy of AC2 and its content was therefore removed.
Citation: https://doi.org/10.5194/egusphere-2023-1490-AC4 -
AC5: 'Reply on RC1', Sara Defratyka, 18 Mar 2024
Publisher’s note: this comment is a copy of AC2 and its content was therefore removed.
Citation: https://doi.org/10.5194/egusphere-2023-1490-AC5
-
RC2: 'Comment on egusphere-2023-1490', Anonymous Referee #2, 13 Jan 2024
Comments on “Statistical evaluation of methane isotopic signatures determined during near-source measurements”
by Defratyka et al.,
The manuscript “Statistical evaluation of methane isotopic signatures determined during near-source measurements” compares different air sampling, measurement and fitting methods to determine δ13CH4 source signatures using Keeling and Miller Tans plots. During a five-day release experiment two mobile laboratories with different sampling and analytical methods were used to measure extensive data sets, which are combined with different data analysis strategy (averaging, linear fitting algorithm) to determine the best methods.
The structure of the manuscript is difficult to follow and large tables with similar values using different fitting algorithms do not contribute to clarity. The authors have not succeeded in focusing on the essentials and highlighting the novelties and value of this study for the scientific community. Results from individual tests, some of which were subject to large measurement uncertainties were used to make general statements about best practice. For such general statements, the collected data should be supplemented with theoretical calculations or synthetic data sets which, as previous studies have shown, are better suited for this purpose.Therefore, I cannot recommend this manuscript for publication in its present form
General:
General conclusions have been drawn and recommendations made based on insufficiently thorough measurement protocols that do not reflect the current state of the art. The CRDS analyzer was only calibrated in the laboratory before it was sent to the campaign and not at regular intervals during the campaign. This is not sufficient for accurate δ13CH4 measurements and is not state of the art for precise measurements. Other studies, e.g. Lopez et al. (2017), Assan et al., (2017) Hoheisel et al. (2019) calibrate similar instruments every other day, every day or twice a day during campaigns.
How does such a calibration strategy affect the results? What is the instrumental drift over time? How long is the time interval between instrument calibration and field measurement?In the conclusion part, CRDS measurements on sample bags are not recommended due to lack of precision. How long have these bags been measured? What was the precision? In the data files orders of magnitude of 3 permil were given. Again, other studies achieve better precision with similar instruments averaging over 20-30 minutes .
The paper Defratyka et al., 2021 (which is not cited in the manuscript, only the data set) used this release campaign to test the capability of the CRDS analyser to measure C2H6. There, the release experiment is well described and gives information about the lengths of each individual release period. Here, it was difficult to follow the actual CH4 release during the different days. In particular, when were the sample bags filled? When were aircores taken? Only in the data file was it possible to see that the 21 IRMS sample bag were filled over the 5 day period. A direct comparison of sample bags and aircore during one release was not presented. Instead a 5 day average was used for the comparison, which does not represent a realistic setup for real field measurements. The authors claim to fill the gap that sampling for isotope signature determination has never been tested with controlled release experiments. However, test of the best sampling strategy: such as how many bag samples or aircore measurements are needed etc. was not presented or evaluated.
The method for linear regressions can be better evaluated with modeled or synthetic data. This has been shown by Wehr and Saleska (2017) for CO2 or Hoheisel et al. (2019) for CH4. In this manuscript, the results of individual regressions on measured data were used to recommend the best approach. Controlling factors such as CH4 excess, difference between source and background 13C values, measurement uncertainties were not systematically investigated or related.
A large part of the paper is devoted to the comparison of the Keeling and Miller Tans methods. It is also quoted that “The Miller-Tans method can be useful to interpret studies, where the Keeling method assumption of stable background is unfulfilled or unknown, e.g. when studies are conducted over a long period of time (Lowry et al., 2020; Al-Shalan et al., 2022).”
The Miller-Tans method (Miller and Tans; 2003) was designed for "longer time series" with changing background conditions, such as multi-year time series and weekly flask samples. . If a year of these data is plotted, one needs to account for the changing background conditions. In case of a 45 min release, the Keeling plot would provide a better estimate of the source signature since we do not expect a changing background over the short period of a release. Most importantly, why would we use the global averaged CH4 mole fraction or a δ13CH4 value from Brownlow et al. (2017) that is one year prior to your experiment? These are unrealistic assumptions that do not improve the approach, but worsen it compared to the Keeling plot, or, if the assumptions happen to hit the right value, appear correct.
Citation: https://doi.org/10.5194/egusphere-2023-1490-RC2 -
AC1: 'Reply on RC2', Sara Defratyka, 18 Mar 2024
Answer to Referee 2 Comments on “Statistical evaluation of methane isotopic signatures determined during near-source measurements” by Defratyka et al.,
A: We would like to thank the reviewer for their valuable suggestions that led to an improvement of the manuscript. In the revised version, we have addressed the reviewer’s comments and have made significant changes to improve the manuscript structure, increase readability and highlight novelty of the study.
R: The manuscript “Statistical evaluation of methane isotopic signatures determined during near-source measurements” compares different air sampling, measurement and fitting methods to determine δ13CH4 source signatures using Keeling and Miller Tans plots. During a five-day release experiment two mobile laboratories with different sampling and analytical methods were used to measure extensive data sets, which are combined with different data analysis strategy (averaging, linear fitting algorithm) to determine the best methods.
The structure of the manuscript is difficult to follow and large tables with similar values using different fitting algorithms do not contribute to clarity. The authors have not succeeded in focusing on the essentials and highlighting the novelties and value of this study for the scientific community. Results from individual tests, some of which were subject to large measurement uncertainties were used to make general statements about best practice. For such general statements, the collected data should be supplemented with theoretical calculations or synthetic data sets which, as previous studies have shown, are better suited for this purpose.A: The purpose of the study was to implement the results of already existing theoretical calculations and synthetic data to real life case studies. The control release experiment is a good representation of the industrial settings and real-life conditions when mobile measurements of methane sources are made. Thus, we conducted an analysis of the available analytical methods applied to real life data, when we can observe the true impact of instrument (frequency, precision), methane range and meteorological conditions. Also, we compared δ13CH4 of the source determined using two different instruments (IRMS vs CRDS) which was not done before. Finally, regarding analytical methods, we focused on methods which were used in previous mobile measurements but not tested before (e.g. BCES linear fitting method, averaging vs clustering into methane range bins, background impact for Miller-Tans method). In the revised manuscript, we have improved the structure of the manuscript to increase the clarity and highlight the novelties of the study. The structure follows the steps presented in the flow chart in Fig. 1. The new figures are plotted to better deliver the message coming from the study and the observed discrepancies between the different analytical strategies. We have introduced a “reference analytical strategy” and the impact of the different analytical steps are presented compared to this reference analytical strategy.
Therefore, I cannot recommend this manuscript for publication in its present form
General:
R: General conclusions have been drawn and recommendations made based on insufficiently thorough measurement protocols that do not reflect the current state of the art. The CRDS analyzer was only calibrated in the laboratory before it was sent to the campaign and not at regular intervals during the campaign. This is not sufficient for accurate δ13CH4 measurements and is not state of the art for precise measurements. Other studies, e.g. Lopez et al. (2017), Assan et al., (2017) Hoheisel et al. (2019) calibrate similar instruments every other day, every day or twice a day during campaigns.How does such a calibration strategy affect the results? What is the instrumental drift over time? How long is the time interval between instrument calibration and field measurement?
A: This particular CRDS model was used during other campaigns like (Assan, 2017; Defratyka et al., 2021a, b). Before implementing the instrument during field campaigns, numerous laboratory tests were conducted to characterize the instrument performance (Defratyka, 2021, chapter 2). For example, the working gas was measured during 21 randomly chosen mobile campaigns during studies focused on Paris agglomeration. This test aimed to check the analyser stability and lack of influence of switching on/off analyser for CH4 and δ13CH4 values. The measured values were stable and did not change over time.
We note that not having the working gas to check the instrument stability is a setback of our studies. However, a three-point calibration of the instrument was made on a regular basis every few months, including one calibration just before the controlled release experiment. Differences between the obtained calibration parameters are statistically insignificant. Additionally to the regular three-point calibration, the working gas was measured just before and after the controlled experiment (one measurement on 06.09.2019 and two measurements on 14.09.2019) in the laboratory. The gas was measured over 20 minutes and after stabilisation last 15, minutes were measured. No significant drift was observed. Based on all mentioned laboratory tests, we are confident that the CH4 and δ13CH4 values are robust and the instrument was stable without significant drift during mobile measurements with AirCore and during bag analysis in the laboratory. In the revised part of the manuscript, we added a better explanation: lines 115-119 “Due to logistic restrictions, the cylinder with calibration or known working gas could not be used during the controlled release experiment. However, an additional working gas was measured in the laboratory day before and day after the experiment and no significant drift was observed. During independent measurement campaigns, additional working gas was measured on used here CRDS before and/or after randomly chosen surveys (Defratyka, 2021, chapter 2). Observed time drift was negligible and working gas measurements inside the vehicle did not improve instrument precision. Thus, 3-point calibration done before controlled release experiment gives robust results. Additionally, the instrument was tested in laboratory conditions to verify possible impact of external pressure and temperature for CRDS instrument. No significant dependence on the ambient pressure and temperature both for CH4 mole fraction and δ13CH4 was observed for CRDS instrument used during this study (Defratyka, 2021, chapter 2). ”
R: In the conclusion part, CRDS measurements on sample bags are not recommended due to lack of precision. How long have these bags been measured? What was the precision? In the data files orders of magnitude of 3 permil were given. Again, other studies achieve better precision with similar instruments averaging over 20-30 minutes .
A: During release experiment, ambient air samples were collected in 5 L bags, which allowed for about 20 minutes of measurements using CRDS G2201-i. After removing initial stabilization time, around 15 minutes of measurements were analysed. Standard deviation was used as uncertainty of both CH4 and δ13CH4. Hoheisel et al (2019) were measuring samples 15 minutes, analysing 10 minutes after instrument stabilization. Thus, shorter time than in our study. They used Allan standard deviation to determine uncertainty of CH4 and δ13CH4, what could potentially explain observed lower uncertainty in their case. Other possible reason why other studies achieve better precision is the dependence of δ13CH4 precision (calculated as standard deviation) from CH4 mole fraction. During previous laboratory experiments (Defratyka, 2021, chapter 2.2.2.2), δ13CH4 precision improved from 3,5‰ to 0,7‰ when CH4 mole fraction of measured cylinder increased from 1,96 ppm to 10 ppm. Similar tendency is observed for bag samples measured in this study. For bag with the lowest CH4 mole fraction (1.96 ppm) δ13CH4 precision achieved 3.5‰, while for bag with the biggest CH4 mole fraction (5.25 ppm), δ13CH4 precision achieved 1.3‰.
R: The paper Defratyka et al., 2021 (which is not cited in the manuscript, only the data set) used this release campaign to test the capability of the CRDS analyser to measure C2H6. There, the release experiment is well described and gives information about the lengths of each individual release period. Here, it was difficult to follow the actual CH4 release during the different days. In particular, when were the sample bags filled? When were aircores taken? Only in the data file was it possible to see that the 21 IRMS sample bag were filled over the 5 day period. A direct comparison of sample bags and aircore during one release was not presented. Instead a 5 day average was used for the comparison, which does not represent a realistic setup for real field measurements. The authors claim to fill the gap that sampling for isotope signature determination has never been tested with controlled release experiments. However, test of the best sampling strategy: such as how many bag samples or aircore measurements are needed etc. was not presented or evaluated.
A: In the revised version of the manuscript, the set-up of controlled release experiment has been better explained to fulfil all referee’s questions.: lines 79-93 “The experiment lasted over 5 days in September 2019 at Bedford Aerodrome, UK. Pure methane was released from a manifolded multi-cylinder pack, of eleven cylinders containing 999.6 ± 10.0 mmol mol-1, with initial pressure about 200 bars. The impurities in cylinders came from ethane (48 ± 10 µmol mol-1) and propane (0.149 ± 0.30 µmol mol-1). All 11 cylinders were filled at the same time from the same CH4 source, ensuring δ13CH4 source remained stable over the entire measurement period. The methane release rate varied between releases, up to 70 L min-1. During the release, CH4 was mixed with ethane (C2H6) in a varying ratio, giving C2H6:CH4 ratios from 0.00 to 0.07. The purity of the C2H6 was 999.9 ± 10.0 mmol mol-1, with impurities mostly from methane (2.27 ± 0.46 µmol mol-1) and propane (7.5 ± 1.5 µmol mol-1). To achieve required CH4 emission rate, CH4 was released simultaneously from all 11 cylinders, using the transportable flow control system (details in Gardiner et al., 2017)). Briefly, flow control system is designed and configured for the creation of ‘real-life’ gaseous emission scenarios, in a range of industrial settings. It allows for validation of methods for emissions monitoring applied during typical field conditions. The control system is built on six mass flow controllers (MFC), (Brooks Instrument, Hatfield, PA,USA). Four primary MFCs provide four independent emission sources, while two secondary MFCs allow for the introduction of purge and interferant gases into the primary flow. The flow control system is computer operated, allowing for the implementation of pre-written operational programs and the post-test analysis (Gardiner et al., 2017).” Additionally to averaged results (daily and 5 days average), the results from one individual release have been compared for CRDS and IRMS: 355-361: “Finally, the results are compared for individual releases where sampling using two different methods was done simultaneously (Tab. 1). For bag samples measured on IRMS and CRDS, simultaneous sampling happened during two releases of first day (release 1.1 and release 1.2). AirCore samples were taking simultaneously with bags for IRMS during third release over third day of the experiment (release 3.3). The residual between δ13CH4 source from bag samples measured on IRMS and CRDS achieved 0.66 ‰ for release 1.1 and 0.21 ‰ for release 1.2. For AirCore sampling residual is even smaller and achieved -0.09 ‰ for release 3.3. For both CRDS measurement techniques, for individual releases, residuals are smaller than uncertainty, and thus results from CRDS and IRMS instrument are in good agreement within the uncertainty (Tab. 1).”
Typically, choice of the best sampling strategy depends on measurements conditions, like source location (e.g. empty road with possibility to stop or traffic jam inside the city) and CH4 enhancement above background (i.e. for CRDS studies higher CH4 mole fraction translate into better precision of measured δ13CH4). Thus, it is difficult to draw general conclusion about the best sampling strategy as it depends on individual measurements campaign and usually it is a compromise between attempt to collect as much samples as possible, and surrounding conditions, including aspects like safety (e.g. stopping car for sampling can cause accident), location and meteorological conditions.
R: The method for linear regressions can be better evaluated with modeled or synthetic data. This has been shown by Wehr and Saleska (2017) for CO2 or Hoheisel et al. (2019) for CH4. In this manuscript, the results of individual regressions on measured data were used to recommend the best approach. Controlling factors such as CH4 excess, difference between source and background 13C values, measurement uncertainties were not systematically investigated or related.
A: Thanks to this comment, we remarked that the novelty of the study was not highlighted enough. Most of previous studies focused on CO2 or and Monte Carlo simulations (despite Hoheisel et al. focused on CH4 studies, also with using of AirCore). Here, controlled release experiment is used as representation of existing methane point sources in a range of typical industrial settings, as described in Gardiner et al. (2017). As studies are focused on mobile measurements, presented results are applicable to all mobile measurements using bag sampling or in situ sampling with AirCore, which are common techniques for δ13CH4 source determination. Here we focused on real life CH4 dataset, obtained from the controlled release experiment, which simulate real life industrial settings, with realistic range of CH4 enhancement, number of collected data points used for fitting line and real instrument uncertainty. Thus, we checked impact of real- life condition for studies of source δ13CH4. Also, we aim for testing all already used fitting approaches from previous studies of mobile CH4 source signature determination and check their potential impact on final result. Some of these methods were already tested, mostly in context of CO2 respiration. Also, we test here BCES method which is used for δ13CH4 source of methane but not tested as profoundly as other methods (OLS, ODR, York).
Also, during the release, methane and ethane were released with varying fluxes and wind speed and direction were measured (added Table A1 in Appendix). Overall, using different measurement methods, we did not observe significant variation of determined δ13CH4, depending on CH4 flow rate, ratio of C2H6: CH4 and wind speed and direction. We included this observation in revised version of manuscript.
Regarding relation between source and background, measurements uncertainties, and impact of number of measurements point, these studies were done previously by Hoheisel et al. (2019) and we were using their conclusion through our studies.
R: A large part of the paper is devoted to the comparison of the Keeling and Miller Tans methods. It is also quoted that “The Miller-Tans method can be useful to interpret studies, where the Keeling method assumption of stable background is unfulfilled or unknown, e.g. when studies are conducted over a long period of time (Lowry et al., 2020; Al-Shalan et al., 2022).”
The Miller-Tans method (Miller and Tans; 2003) was designed for "longer time series" with changing background conditions, such as multi-year time series and weekly flask samples. . If a year of these data is plotted, one needs to account for the changing background conditions. In case of a 45 min release, the Keeling plot would provide a better estimate of the source signature since we do not expect a changing background over the short period of a release. Most importantly, why would we use the global averaged CH4 mole fraction or a δ13CH4 value from Brownlow et al. (2017) that is one year prior to your experiment? These are unrealistic assumptions that do not improve the approach, but worsen it compared to the Keeling plot, or, if the assumptions happen to hit the right value, appear correct.
A: During mobile measurements we observe also a short term change of background CH4 mole fraction. In the morning, after night-time accumulation, CH4 mole fraction can be higher than usually and decrease with time, what was observed especially during winter’s months. Moreover, δ13CH4 of sources are compared between each other in different locations and to observe possible changes over time (e.g., seasons, years). Thus, Miller-Tans method is more suitable for mobile measurements even though there are short-term studies of one individual source. As some groups use or Miller-Tans or Keeling method, we compared results of both of them to see if any method implement bias and can affect comparison of results from different groups.
We used global values of CH4 and δ13CH4 as an additional random background to see how differently chosen backgrounds can affect the results. Notably, during mobile measurements, researcher can face different conditions, when local enhancement of CH4 can be observed and they face the decision of choosing the background. Thus, we wanted to show how changes in background can directly affect determined δ13CH4 of methane source. In the revised version of the manuscript, we have improved the explication: lines 211-219 “Next, to verify the sensitivity of Miller-Tans method for a differently defined background, calculations for two randomly chosen backgrounds with lower CH4 bckg mole fraction and δ13CH4 bckg than during the experiment were conducted: “random I” and “random II” background. For random I background, CH4 bckg mole fraction is defined as an average global CH4 mole fraction observed in September 2019, equals to 1.8707 ± 0.0011 µmol mol-1 (NOAA/ESRL), while δ13CH4 bckg is defined using value from Brownlow et al., as -47.2 ± 0.2 ‰ (2017), to verify if slight difference in background can translate into significant differences in determined δ13CH4 source. For random II background, the CH4 bckg mole fraction is set up the same as the random I background, but the δ13CH4 bckg was randomly set to -42.7 ± 0.2 ‰ to significantly differ from other δ13CH4 bckg backgrounds to better test the sensitivity of Miller-Tans method to subtracted background.” We have also rewritten the manuscript and updated the figures to highlight differences between all used strategies not only to focus on comparison between Keeling and Miller-Tans methods.
Citation: https://doi.org/10.5194/egusphere-2023-1490-RC2
References:
Assan, S.: Towards improved source apportionment of anthropogenic methane sources, Paris-Saclay, Gif sur Yvette, 160 pp., 2017.
Brownlow, R., Lowry, D., Fisher, R. E., France, J. L., Lanoisellé, M., White, B., Wooster, M. J., Zhang, T., and Nisbet, E. G.: Isotopic Ratios of Tropical Methane Emissions by Atmospheric Measurement: Tropical Methane δ13C Source Signatures, Glob. Biogeochem. Cycles, 31, 1408–1419, https://doi.org/10.1002/2017GB005689, 2017.
Defratyka, S.: Characterizing methane (CH4) emissions in urban environments (Paris), Paris-Saclay, Gif sur Yvette, 185 pp., 2021.
Defratyka, S. M., Paris, J.-D., Yver-Kwok, C., Loeb, D., France, J., Helmore, J., Yarrow, N., Gros, V., and Bousquet, P.: Ethane measurement by Picarro CRDS G2201-i in laboratory and field conditions: potential and limitations, Atmospheric Meas. Tech., 14, 5049–5069, https://doi.org/10.5194/amt-14-5049-2021, 2021a.
Defratyka, S. M., Paris, J.-D., Yver-Kwok, C., Fernandez, J. M., Korben, P., and Bousquet, P.: Mapping Urban Methane Sources in Paris, France, Environ. Sci. Technol., 55, 8583–8591, https://doi.org/10.1021/acs.est.1c00859, 2021b.
NOAA/ESRL: https://gml.noaa.gov/ccgg/trends_ch4/.
Gardiner, T., Helmore, J., Innocenti, F., and Robinson, R.: Field Validation of Remote Sensing Methane Emission Measurements, Remote Sens., 9, 956, https://doi.org/10.3390/rs9090956, 2017.
Hoheisel, A., Yeman, C., Dinger, F., Eckhardt, H., and Schmidt, M.: An improved method for mobile characterisation of D13CH4 source signatures and its application in Germany, Atmospheric Meas. Tech., 12, 1123–1139, https://doi.org/10.5194/amt-12-1123-2019, 2019.
Citation: https://doi.org/10.5194/egusphere-2023-1490-AC1
-
AC1: 'Reply on RC2', Sara Defratyka, 18 Mar 2024
Status: closed
-
RC1: 'Comment on egusphere-2023-1490', Anonymous Referee #1, 04 Sep 2023
The authors of the manuscript ‘Statistical evaluation of methane isotopic signatures determined during near-source measurements’ compare the effects from various measurement techniques and analytical methods on δ13C-CH4 source values within a controlled gas release experiment. They intend to give generalized recommendations for the best practice in determing δ13C-CH4 source values during mobile measurements.
In general, because of the controlled conditions, the experimental set-up provides a good basis for studying the effects of near-source measurements. However, I think that the set-up is more suitable for studies that focus on the comparison of measurement methods and the change of environmental conditions such as distance to the source or changes in fluxes.
Comparing different analysis methods (mass conservation method, linear fitting, averaging) is a good idea in principle and could be helpful in establishing best practice guidelines. However, the analysis methods observed here are potentially better suited for computational/modeling studies in which the dominant effects are studied in the context of a multivariate approach.
All in all, I would not recommend this manuscript for publication at the present stage due to the reasons addressed in the general and specific comments below:
general comments:
First of all, the structure of the manuscript is quite poor. For the reader, the content is often confusing and difficult to follow, as basic information, results, discussion and conclusion are wildly mixed in the different manuscript sections and also between the main part of the manuscript and the appendix (see specific comments). Especially because of the numerous analysis parameters to be considered, it is important to create a clear structure, so that the reader does not lose the overview. In this context, Figure 1 is very helpful. It should therefore be used as a "guide" through the manuscript. For example, the analytical methods as well as the results could be presented in the same order as shown in Figure 1 using the same titles, such as: 1. mass conservation method, 2. background subtraction, ... And, similar, for the results: 1. effects from mass conservation, 2. effects from different backgrounds, ...In addition, the various results sections should each contain a corresponding figure showing a comparison of one single parameter. This will contribute to greater clarity and conciseness. A minor point is the clear distinction between the observed δ13CH4 and source δ13CH4 values by using different indices such as ‘o’, ‘s’.
The title is somewhat misleading as it implies a dominance of statistical analysis. However, the manuscript describes more of an application study, investigating numerous parameters based on near-source laboratory measurements and comparing different analytical methods such as mass conservation method or linear fitting method. From the title, I would have expected a manuscript on multivariate statistics to understand the relationships between the numerous parameters and their importance in determining isotopic source values.
Another important criticism is that the novelty of the study is not obvious. The results presented show more or less similar results to previous studies. There are already several publications comparing the different linear fitting methods and mass conservation methods that conclude that the Miller-Tans-Method is the best method when the background value varies, and that the York method is the best fitting method for uncertainties in both, x as well as in y. Some of the results presented here are already self-evident, such as that CRDS uncertainties are higher than those of IRMS measurements due to lower instrument precision.
In my opinion, the requirements for a best-practice recommendation, which the authors have set as their goal, are not met. The given requirements for a generally valid analysis concept are not fulfilled, since the measurements are performed under very specific conditions (synthetic/laboratory) and several problems have not yet been solved, such as the possible temperature effects of the direct measurements or the depletion of CRDS values compared to IRMS values (potential calibration effects).
specific comments:
Abstract:
- line 20: The issues ‘accessibility, practicality and costs’ are not addressed further in the manuscript. I would have expected these topics to be included in the main part as well.
- general: normally, the abstract is a brief summary of the research presented and therefore should also include the main findings of the study, which is what is missing here.
- Introduction:
- line 58: To me, the phrase ‘sampling over five days’ implies a time series. Since you are working with an artificial source that should provide constant values over time, I would rather speak of ‘five consecutive runs’.
- line 62-67: this section rather belongs to the chapter conclusion/outlook
- Controlled release experiment and sampling methodology:
- controlled release set up:
- what is the pressure and volume of the gas cylinders?
- at which location does the release experiment take place?
- is there a control of wind speed and fluxes?
- a more detailed description (besides mentioning of Gardiner et al.) of the controlled release system would be beneficial in this section, as it is the core of the experiment. A graphical scheme would also be helpful.
- Does the decreasing gas pressure of the 12 cylinders have an effect on the isotopic values, since we have sometimes observed a slight change in isotope values with decreasing gas pressure in our standard gas bottles?
- line 68: I would give chapter 2 a more general title like ‘experimental setup’. As the mass conservation method is also an analytical method I would include subchapter 2.4 into chapter 3
- line 80: The use of ‘‰’ to express isotope quantities is obsolete. It is no longer encouraged by IUPAC. According to Brand and Coplen (Assessment of international reference materials for isotope-ratio, 2014), the term ‘milli-Urey’ (mUr) should replace the old ‰ sign. See Werner and Cormier (‘Isotopes-Terminology, Definitions and Properties’, 2022) for current terminology.
- line 83: why was the direct sample from the cylinders not taken both at the beginning and at the end of the experiment to rule out effects such as fluctuating gas pressure?
- line 85: IRMS is not yet explained- please provide a short sentence on instrument specification or a link to the corresponding chapter.
- line 91-105: here the link to Appendix A is missing, where the methods are explained in detail.
- line 110: did I understand this correctly: the Picarro data from the RHUL mobile laboratory was not used for continuous measurement analysis, but only for peak detection for the bag samples? Why not using here an AirCore sampling technique as well?
- Analytical methods of the acquired measurements
- line 160-162: I do not understand that sentence.
- line 162-165: why do you not specify a wide range of different random backgrounds for sensitivity analysis instead of using ‘average’, ‘global’ and ‘random’ backgrounds, some of which are very close to each other?
- line 181: define ‘R’ for the reader who is not familiar with programing languages.
- line 181: what is the mathematical difference between OLS II and MA, since you use the same command in R (lmodel2())?
- line 199: Does it even make sense to test a method other than the one that takes into account the error in x and y such as York and BCES-regression method? Because that is what you would expect from your data which is biased in x as well as in y.
- subchapter 3.5: For a better overview, this subchapter should definitely be placed at the beginning of Chapter 3.
- Results
- line 268-270: A reference for that observation is missing? Table 1?
- line 272: I do not think, that the results are the same. I would rather speak of ‘similar results’.
- line 288-295: this section belongs to the chapter discussion/conclusion
- line 299-300: I did not find information on temperature instability and cavity pressure in Appendix B
- line 301-302: would it be not more appropriate to compare the AirCore values with the CRDS values from Table 2, as both belong to the same method? Or, if you want to use a reference value, why not use those values measured directly from the cylinder because this is more or less the ‘true source’ of δ13C-values?
- line 311 ff: this paragraph is an interpretation and should not be part of the results section
- line 316-321: here, a figure would be helpful to emphasize the results.
- line 321,330 ff: recommendations should be no part of the results section.
- line 339 ff.: this is again an interpretation
- line 351-354: this information is redundant, as it can already be found in subchapter 2.2.
- line 368-369: I do not believe that temperature effects play such a large role in the observed isotope discrepancies. Since direct and indirect measurements are based on completely different conditions (differences in fluxes, flows, mixing effects), I would not have expected unbiased values (deviation from the ‘true’ direct measurements) for the indirectly measured isotope values.
- Discussion
- line 384: The analytical technique used depends primarily on the environmental conditions and requirements, and on the sensitivity required (relative or absolute measurements). For field measurements, it is not always possible to take bag samples and store them for IRMS measurements.
- line 388: The Miller-Tans-method should be used in any case where the background varies (usually in long-term studies). If the background is constant or not measured, the Keeling method can be used. Accordingly, there is not really a question which of the two methods is the better one, because the application depends on measurement conditions.
- line 390: Hoheisel et al. (2019) seem to come to different conclusions: “Especially for natural gas samples, the precise determination and correction of C2H6 is important as in our study C2H6 can bias 13CH4 by up to 3‰ depending on the CH4-to-C2H6 ratio of the sample and the calibration cylinder.” How can you explain this discrepancy?
- line 400-402: there are some more studies for example Takriti et al. (‘Mobile methane measurements: Effects of instrument specifications on data interpretation, reproducibility, and isotopic precision’, 2021) and Hoheisel et al. (2019). So why are you focusing only on the studies mentioned in line 401-402 to compare your results?
- line 415: not only Wehr and Saleska proposed York fitting method, but also many other studies come to the same conclusion such as for example Hoheisel et al. (2019)
- line 420-422: Wehr and Saleska performed model simulations within their study. It is difficult to compare this result with the results of a field test conducted as part of this study.
- line 424-427: what is the conclusion from that statement? Instrument settings have not been discussed before, and introducing another parameter at this point is inappropriate. Therefore, I see no benefit in mentioning this here.
- line 428-429: this is a self-evident fact that applies to all occasions that require sample dilution.
- line 430: I would rather speak of expectable then remarkable (see last comments on results). In general, possible fractionation effects should be checked before starting the actual experiment: e.g. due to different wind speeds, different inflow angles, different temperatures etc. As long as the experimental framework conditions are not clarified, an experiment that investigates numerous parameters is not useful, in my opinion.
- line 436-439: I consider this statement questionable as the experimental conditions are very specific (artificial release of gases to the atmosphere).
- Conclusion
- line 447: I would not speak of statistical methods, but rather of a comparison under different methodological and analytical aspects
Appendix
- Appendix B should appear in a shortened version in the main part of the manuscript (one to two sentences)
- At the end of Appendix B a title (‘Results for bag samples measured on IRMS and CRDS for all examined linear fitting methods’) appears without text following
- Appendix C: A figure would help to clarify the impact of data clustering. I would include this section in the main part of the manuscript.
- for the mobile sampling laboratories, a table with specifications would be beneficial for a better overview
Figures and Tables
- Figure 1: The figure is very clear and helpful to the reader. However, I miss the applied mobile laboratory and the measured components (CH4, δ13CH4) as a note in the graph
- Figure 1: inconsistency: the averaged AirCore samples are presented in ppb- in the main part they are given in nmol/mol
- Figure 3: plotting different CH4 mole fractions based on the size of the data points is absolutely confusing - rather use an average value. A general advice: do not show more than one dependency in one figure
- Figure 3: what does ‘release’ mean? Does this refer to the day of release or to the run number?
- Figure 3: Do you really want to include the data from release 4-5, which is, as you mentioned in the text, biased by calibration errors? Since you also omit the data for averaging I would not show it at all as it confuses the reader.
- Figure B1, C1: these figures are not mentioned anywhere in the text!
- Figure C1: use similar y-axis for better comparison
- Table 3: probably transposed numbers for MA- δ13CH4 Keeling method? (-42.18 instead of -24.18?)
technical corrections:
- please check again for correct super/subscript (i.e., CH4 instead of CH4) throughout the whole manuscript as there are many discrepancies
- line 130 ff: use consistently the term CH4s for source values to distinguish source values from observed values (see formula 1)
- line 31: delete the word ‘measurements’
- line 46: better write ‘equipped with an atmospheric sampling system (AirCore)’ as you have not introduced the AirCore system yet
- line 51: ‘determine δ13C source values’
- line 146: delete commas in that sentence
- line 265-267: ensure that the formulas are displayed correctly: μmol∙mol-1
- line 268: space between treatmen t
- line 339-340: rewrite the sentence as it is not clear (3 times the word corrections!)
-
AC2: 'Reply on RC1', Sara Defratyka, 18 Mar 2024
Answer to Referee 1 Comments on “Statistical evaluation of methane isotopic signatures determined during near-source measurements” by Defratyka et al.,
A: We would like to thank the reviewer for their valuable suggestions that led to an improvement of the paper. In the revised version of the manuscript the structure is improved based on the reviewer’s suggestions and the reviewer’s comments are addressed.
R: The authors of the manuscript ‘Statistical evaluation of methane isotopic signatures determined during near-source measurements’ compare the effects from various measurement techniques and analytical methods on δ13C-CH4 source values within a controlled gas release experiment. They intend to give generalized recommendations for the best practice in determing δ13C-CH4 source values during mobile measurements.
In general, because of the controlled conditions, the experimental set-up provides a good basis for studying the effects of near-source measurements. However, I think that the set-up is more suitable for studies that focus on the comparison of measurement methods and the change of environmental conditions such as distance to the source or changes in fluxes.
A: Thank you for highlighting this aspect of our study. During this controlled release experiment, three measurement methods were used and compared (bag samples measured in the laboratory using IRMS or CRDS and AirCore with CRDS in situ sampling). In the revised version, we better highlight this part of our studies. Also, during the release, methane and ethane were released with varying fluxes and wind speed and direction were measured (added Table A1 in Appendix). Overall, using different measurement methods, we did not observe any significant variation of determined δ13CH4, depending on CH4 flow rate, ratio of C2H6: CH4 or wind speed and direction. We included this observation in the revised version of the manuscript.
R: Comparing different analysis methods (mass conservation method, linear fitting, averaging) is a good idea in principle and could be helpful in establishing best practice guidelines. However, the analysis methods observed here are potentially better suited for computational/modeling studies in which the dominant effects are studied in the context of a multivariate approach.
A: Most of previous studies focused on CO2 or and Monte Carlo simulations (despite Hoheisel et al. focusing on CH4 studies, and using AirCore). Here we focused on real life CH4 dataset, obtained from the controlled release experiment, which simulates real life industrial settings, with a realistic range of CH4 enhancement, number of collected data points used for fitting line and real instrument uncertainty. Thus, we checked the impact of real-life condition for studies of source δ13CH4. Also, we aimed to test all already used fitting approaches from previous studies of mobile CH4 source signature determination and check their potential impact on the final result. Some of these methods were already tested, mostly in context of CO2 respiration. Also, we test here the BCES method which is used for δ13CH4 source of methane but not tested as profoundly as other methods (OLS, ODR, York).
R: All in all, I would not recommend this manuscript for publication at the present stage due to the reasons addressed in the general and specific comments below:
General comments:
R: First of all, the structure of the manuscript is quite poor. For the reader, the content is often confusing and difficult to follow, as basic information, results, discussion and conclusion are wildly mixed in the different manuscript sections and also between the main part of the manuscript and the appendix (see specific comments). Especially because of the numerous analysis parameters to be considered, it is important to create a clear structure, so that the reader does not lose the overview. In this context, Figure 1 is very helpful. It should therefore be used as a "guide" through the manuscript. For example, the analytical methods as well as the results could be presented in the same order as shown in Figure 1 using the same titles, such as: 1. mass conservation method, 2. background subtraction, ... And, similar, for the results: 1. effects from mass conservation, 2. effects from different backgrounds, ...In addition, the various results sections should each contain a corresponding figure showing a comparison of one single parameter. This will contribute to greater clarity and conciseness. A minor point is the clear distinction between the observed δ13CH4 and source δ13CH4 values by using different indices such as ‘o’, ‘s’.
A: Thank you for this important remark. For the revised version, the structure of paper has been improved based on the referee’s comment, leaving the most crucial information and removing less important results. The structure has been corrected based on the suggestion of the reviewer, following the flow chart presented in Figure 1, both for methods and results sections. Section 3.5 has been moved at the beginning of section 3 for better guidance through the paper. The figures and tables have been improved too. Also subscripts ‘obs’ and ’source’ have been added to distinguish observed and source δ13CH4.
R: The title is somewhat misleading as it implies a dominance of statistical analysis. However, the manuscript describes more of an application study, investigating numerous parameters based on near-source laboratory measurements and comparing different analytical methods such as mass conservation method or linear fitting method. From the title, I would have expected a manuscript on multivariate statistics to understand the relationships between the numerous parameters and their importance in determining isotopic source values.
A: Based on the reviewer’s suggestion, we have removed “statistical” from the title, leaving: “Evaluation of methane isotopic signatures determined during near-source measurements”.
R: Another important criticism is that the novelty of the study is not obvious. The results presented show more or less similar results to previous studies. There are already several publications comparing the different linear fitting methods and mass conservation methods that conclude that the Miller-Tans-Method is the best method when the background value varies, and that the York method is the best fitting method for uncertainties in both, x as well as in y. Some of the results presented here are already self-evident, such as that CRDS uncertainties are higher than those of IRMS measurements due to lower instrument precision.
A: Thanks to this comment, we remarked that the novelty of the study was not highlighted enough. In this study, we showed how the lower/ higher precision of instrument translates into a determined source δ13CH4. We focused on CH4 studies, from mobile measurements where number of points and enhancement range are different from CO2 respiratory conditions, which was the main focus of previous studies. Additionally, we included not previously tested BCES, which is another method which includes explicit uncertainty of x and y. It allowed for comparison of two methods with uncertainties (York and BCES), what was not done before. We also checked the impact of used background and averaging for the result. Overall, we tested the impact of the decisions made by a researcher during data analysis on the final δ13CH4 source. Our approach is a validation and implementation of previous results for case of mobile indirect measurements of δ13CH4. The manuscript has been revised accordingly to highlight the importance and novelty of the study: line 64-75: “To derive a more universal analytical approach for near-source studies of δ13CH4 source, isotopic measurement and samples collection were included within a controlled release experiment, which represents real- life emissions from existing methane sources, like natural gas infrastructure or sewage emission. The experiment focused on the validations of methods already applied during mobile, vehicle-based methane measurements (e.g., Hoheisel et al., 2019; Defratyka et al., 2021; Fernandez et al., 2022; Menoud et al., 2022). Samples collected over five consecutive days of the experiment were used to compare Isotope Ratio Mass Spectrometry (IRMS) and Cavity Ring Down Spectroscopy (CRDS) measurement techniques (Sect. 2.2), while CRDS instrument was used both for in-situ sampling using AirCore tool, and for remote bag analysis after the sample collection. Comparing these three types of measurement technique, we provide insight into results from high precision (IRMS) and low precision (CRDS) instruments and impact of increasing data frequency for low precision instrument (CRDS with AirCore). Moreover, the studies were focused on a comprehensive inter-comparison of Keeling and Miller-Tans methods (Sect. 3.1), the impact of chosen backgrounds for Miller-Tans method (Sect. 3.2) and impact of averaging clusters (Sect. 3.3). Finally, data were re-analysed using different linear fitting methods (Sect. 3.4).” and: 436-444: “The novelty of the study is the comprehensive inter-comparison between indirect studies of δ13CH4 using (i) bag sampling measured afterwards both by IRMS and CRDS, (ii) in-situ CRDS with an AirCore storage tool under controlled release conditions. Also, we focused on intercomparison of different analytical strategies already used in indirect studies of δ13CH4 source, which was not done before on such a broad scale. We tested aspects which were not detailed in previous studies, like background subtraction, data averaging and BCES linear fitting. To achieve it, we focused on data from controlled release experiment, which is a representation of real-life point methane sources, like for example leaks in natural gas infrastructure. This approach allows for validation and implementation of conclusions from studies mostly focused on synthetic or CO2 data (Pataki et al., 2003; Miller and Tans, 2003; Zobitz et al., 2006; Wehr and Saleska, 2017).”
R: In my opinion, the requirements for a best-practice recommendation, which the authors have set as their goal, are not met. The given requirements for a generally valid analysis concept are not fulfilled, since the measurements are performed under very specific conditions (synthetic/laboratory) and several problems have not yet been solved, such as the possible temperature effects of the direct measurements or the depletion of CRDS values compared to IRMS values (potential calibration effects).
A: Overall, controlled release experiment is used as a representation of existing methane point sources in a range of typical industrial settings, as described in Gardiner et al. (2017). As studies are focused on mobile measurements, presented results are applicable to all mobile measurements using bag sampling or in situ sampling with AirCore, which are common techniques for δ13CH4 source determination. In the revised version of the manuscript, we have changed from the general best practice recommendations to recommendation for case studies of mobile measurements of CH4.
Regarding potential discrepancies between CRDS and IRMS results, numerous testes of CRDS instruments were done and details can be found in (Defratyka, chapter 2, 2021). No significant impact of temperature or pressure for CRDS results was observed. δ13CH4 of used calibration cylinders was assigned at RHUL, ensuring measurements on the same scale. In the revised manuscript, the following text has been added: lines 115-124 “Due to logistic restrictions, the cylinder with calibration or known working gas could not be used during the controlled release experiment. However, an additional working gas was measured in the laboratory the day before and the day after the experiment and no significant drift was observed. During other independent measurement campaigns, an additional working gas was measured on the CRDS used here, before and/or after randomly chosen surveys (Defratyka, 2021, chapter 2). Observed time drift was negligible and working gas measurements inside the vehicle did not improve instrument precision. Thus, 3-point calibration done before controlled release experiment gives robust results. Additionally, the instrument was tested in laboratory conditions to verify possible impact of external pressure and temperature for CRDS instrument. No significant dependence on the ambient pressure and temperature both for CH4 mole fraction and δ13CH4 was observed for CRDS instrument used during this study (Defratyka, 2021, chapter 2).”
specific comments:
Abstract:
- line 20: The issues ‘accessibility, practicality and costs’ are not addressed further in the manuscript. I would have expected these topics to be included in the main part as well.
A: In the revised version of the manuscript, “accessibility, practicality and costs” have been removed from the abstract.
- general: normally, the abstract is a brief summary of the research presented and therefore should also include the main findings of the study, which is what is missing here.
A: An additional part with main finding of the study have been included in the abstract in the revised version: “Due to high precision and accuracy of IRMS instruments, the chosen analytical strategy does not significantly affect IRMS results. However more precautions must be made for analysis of CRDS AirCore samples. Fitting methods with forced symmetry like Major Axis or Bivariate Correlated Errors and Intrinsic Scatter (BCES) Orthogonal introduce significant biases in the determined isotopic signatures of methane sources. The most reliable results are obtained for non-averaged data using fitting methods, which include uncertainties on x- and y-axis values, like York fitting or BCES (Y|X). Ordinary Least Squares method provides sufficiently robust results and, for simplicity, can be used to determine δ13CH4 in near-source conditions.”
Introduction:
- line 58: To me, the phrase ‘sampling over five days’ implies a time series. Since you are working with an artificial source that should provide constant values over time, I would rather speak of ‘five consecutive runs’.
A: the phrase has been corrected in the revised version.
- line 62-67: this section rather belongs to the chapter conclusion/outlook
A: This part has been removed from revised version.
Controlled release experiment and sampling methodology:
- controlled release set up:
- what is the pressure and volume of the gas cylinders?
- at which location does the release experiment take place?
- is there a control of wind speed and fluxes?
- a more detailed description (besides mentioning of Gardiner et al.) of the controlled release system would be beneficial in this section, as it is the core of the experiment. A graphical scheme would also be helpful.
- Does the decreasing gas pressure of the 12 cylinders have an effect on the isotopic values, since we have sometimes observed a slight change in isotope values with decreasing gas pressure in our standard gas bottles?
A: In revised version of the manuscript, description of controlled experiment set up has been more detailed, to answer all Referee’s questions: lines 79-93: “The experiment lasted over 5 days in September 2019 at Bedford Aerodrome, UK. Pure methane was released from a manifolded multi-cylinder pack, of eleven cylinders containing 999.6 ± 10.0 mmol mol-1, with initial pressure about 200 bars. The impurities in cylinders came from ethane (48 ± 10 µmol mol-1) and propane (0.149 ± 0.30 µmol mol-1). All 11 cylinders were filled at the same time from the same CH4 source, ensuring δ13CH4 source remained stable over the entire measurement period. The methane release rate varied between releases, up to 70 L min-1. During the release, CH4 was mixed with ethane (C2H6) in a varying ratio, giving C2H6:CH4 ratios from 0.00 to 0.07. The purity of the C2H6 was 999.9 ± 10.0 mmol mol-1, with impurities mostly from methane (2.27 ± 0.46 µmol mol-1) and propane (7.5 ± 1.5 µmol mol-1). To achieve the required CH4 emission rate, CH4 was released simultaneously from all 11 cylinders, using the transportable flow control system (details in Gardiner et al., 2017)). Briefly, the flow control system is designed and configured for the creation of ‘real-life’ gaseous emission scenarios, in a range of industrial settings. It allows for the validation of methods for emission monitoring applied during typical field conditions. The control system is built on six mass flow controllers (MFC), (Brooks Instrument, Hatfield, PA, USA). Four primary MFCs provide four independent emission sources, while two secondary MFCs allow for the introduction of purge and interferant gases into the primary flow. The flow control system is computer operated, allowing for the implementation of pre-written operational programs and the post-test analysis (Gardiner et al., 2017).”
Regarding the question of decreasing gas pressure of the 12 cylinders, we do not expect any effect from the decrease of the pressure on δ13CH4. For clarification, in the first version of manuscript we wrote 12 cylinders instead of 11 cylinders. First, the gas was released from 11 cylinders in the batch simultaneously, using controlling station. Second, to verify the possibility of δ13CH4 fractionation during the experiment, last summer during an additional control release experiment, three samples were taken directly from cylinder batch, similar to way it was done during our controlled release experiment. First sample was taken when cylinder pack was first used. Second sample was taken immediately after we had been withdrawing >50 litres/min from it for 1 hour. Cylinders slightly cool, pressure regulator iced. And third sample was taken on the last day of tests but before any tests had started for the day. No statistically significant differences were observed. This reject the hypothesis of decreasing gas pressure of the 11 cylinders having an effect on the isotopic values and also the hypothesis about the potential impact of temperature for fractionation between direct and indirect sampling, which was initially mentioned in the first version of the manuscript and has been removed for revised version.
- line 68: I would give chapter 2 a more general title like ‘experimental setup’. As the mass conservation method is also an analytical method I would include subchapter 2.4 into chapter 3
A: The proposed changes have been implemented in the revised version.
- line 80: The use of ‘‰’ to express isotope quantities is obsolete. It is no longer encouraged by IUPAC. According to Brand and Coplen (Assessment of international reference materials for isotope-ratio, 2014), the term ‘milli-Urey’ (mUr) should replace the old ‰ sign. See Werner and Cormier (‘Isotopes-Terminology, Definitions and Properties’, 2022) for current terminology.
A: The “‰” is still widely used in the atmospheric community to express the isotopic ratio of δ13CH4. For example, results of flask samples presented at NOAA website are also expressed in ‰ unit. Also, recent papers focusing on continuous measurements of δ13CH4 use ‰ unit (Hoheisel and Schmidt, 2023; Maisch et al., 2023). Moreover, broadly used international guidelines also use ‰ (Coplen, 2011, cited <1300 times; Brand et al., 2014, cited <500 times). Also, CCQM working Group on Isotope Ratios in their strategy for 2021-2030 uses ‰ (CCQM, 2021). Additionally, readers are much more familiar with ‰ unit than with new proposed milli-Urey unit. Thus, respectfully, we will stay with ‰ unit, as in our opinion it is more suitable for our study.
- line 83: why was the direct sample from the cylinders not taken both at the beginning and at the end of the experiment to rule out effects such as fluctuating gas pressure?
A: We did not expect observing changes of isotopic signature during release thus we took a direct sample only at the end of the experiment. However, as obtained result posed question about potential impact of gas pressure in cylinder and temperature of released gas for δ13CH4, during another controlled release experiment (summer 2023), we took three samples directly from the multi-cylinder pack, as explained above. No statistically significant differences were observed. This rejects the hypothesis of decreasing gas pressure of the 11 cylinders influencing the isotopic values during our controlled release experiment.
- line 85: IRMS is not yet explained- please provide a short sentence on instrument specification or a link to the corresponding chapter.
A: In revised version of the manuscript, description of bag samples measured on IRMS (Sect. 2.2.1) has been moved before explication of direct sampling (2.2.4), and explication of the instrument is in section 2.2.1.
- line 91-105: here the link to Appendix A is missing, where the methods are explained in detail.
A: A link has been added in revised version.
- line 110: did I understand this correctly: the Picarro data from the RHUL mobile laboratory was not used for continuous measurement analysis, but only for peak detection for the bag samples? Why not using here an AirCore sampling technique as well?
A: Yes, that is correct. RHUL mobile laboratory uses Picarro model G2301 which does not measure δ13CH4. The information has been added in the manuscript, and table with used instrument specification has been added in Appendix: lines 103-105 “This vehicle was equipped with a Picarro CRDS G2301 analyser, capable to measure mole fraction of CO2, CH4 and H2O, a Los Gatos Research Ultraportable Methane Ethane Analyzer (LGR UMEA) and a manually operated diaphragm pump for air sample bag filling (Table A2).”
Analytical methods of the acquired measurements
- line 160-162: I do not understand that sentence.
A: The sentence has been revised for clarification: : “For random I background, CH4 bckg mole fraction is defined as an average global CH4 mole fraction observed in September 2019, equals to 1.8707 ± 0.0011 µmol mol-1 (NOAA/ESRL), while δ13CH4 bckg is defined using value from Brownlow et al., as -47.2 ± 0.2 ‰ (2017), to verify if slight difference in background can translate into significant differences in determined δ13CH4 source.”
- line 162-165: why do you not specify a wide range of different random backgrounds for sensitivity analysis instead of using ‘average’, ‘global’ and ‘random’ backgrounds, some of which are very close to each other?
A: This part has been improved in revised manuscript: lines 211-219 “Next, to verify the sensitivity of Miller-Tans method for a differently defined background, calculations for two randomly chosen backgrounds with lower CH4 bckg mole fraction and δ13CH4 bckg than during the experiment were conducted: “random I” and “random II” background. For random I background, CH4 bckg mole fraction is defined as an average global CH4 mole fraction observed in September 2019, equals to 1.8707 ± 0.0011 µmol mol-1 (NOAA/ESRL), while δ13CH4 bckg is defined using value from Brownlow et al., as -47.2 ± 0.2 ‰ (2017), to verify if slight difference in background can translate into significant differences in determined δ13CH4 source. For random II background, the CH4 bckg mole fraction is set up the same as the random I background, but the δ13CH4 bckg was randomly set to -42.7 ± 0.2 ‰ to significantly differ from other δ13CH4 bckg backgrounds to better test the sensitivity of Miller-Tans method to subtracted background.”
- line 181: define ‘R’ for the reader who is not familiar with programing languages.
A: It has been rewritten as: line 262 “Most of the tested fitting are calculated using built in packages and functions in R programming language…”
- line 181: what is the mathematical difference between OLS II and MA, since you use the same command in R (lmodel2())?
A: Function lmodel2() allows to calculate slope and intercept of linear fitting using different fitting methods: OLS, MA, SMA (standard major axis) and RMA (ranged major axis). Thus, OLS II should give the same results as lm() function in R which calculates slope and intercept only using OLS method. As expected, these two functions give the same results for OLS fitting, although in some cases differences with uncertainties were observed. In revised version of the manuscript, we have removed all OLS II calculations for clarification of the manuscript, as adding this fitting does not bring any additional strategy comparison and it is rather internal R language packages comparison.
- line 199: Does it even make sense to test a method other than the one that takes into account the error in x and y such as York and BCES-regression method? Because that is what you would expect from your data which is biased in x as well as in y.
A: We decided to include also methods which do not take into account errors in x and y (OLS and MA) as they were used in previous studies for determination of source signature (e.g. Defratyka, 2021; Menoud et al., 2022; Takriti et al., 2023). Additionally, even if methods like York and BCES exist, OLS is still the most used fitting method. Thus, including OLS and MA allows for testing potential bias implemented by fitting methods without errors.
- subchapter 3.5: For a better overview, this subchapter should definitely be placed at the beginning of Chapter 3.
A: The section 3.5 has been moved to the beginning of section 3.
Results
- line 268-270: A reference for that observation is missing? Table 1?
A: The reference to Figure 3 will be added in revised version.
- line 272: I do not think, that the results are the same. I would rather speak of ‘similar results’.
A: “the same results” has been replaced by “similar results”.
- line 288-295: this section belongs to the chapter discussion/conclusion
A: This part has been removed from the manuscript as it does not belong to results and it is repetition of already existing discussion.
- line 299-300: I did not find information on temperature instability and cavity pressure in Appendix B
A: The information about temperature and cavity pressure instability was included in Appendix A instead of Appendix B (lines 593-596). The correct appendix has been mentioned in the revised version.
- line 301-302: would it be not more appropriate to compare the AirCore values with the CRDS values from Table 2, as both belong to the same method? Or, if you want to use a reference value, why not use those values measured directly from the cylinder because this is more or less the ‘true source’ of δ13C-values?
A: The three measurement method have been compared in the same section (Sect. 2.2 for methods and 4.1 for results), following the order presented in flow chart on Figure 1, for easier following by the reader. For better clarification, we defined “reference analytical strategy: lines 312-314: “For comparison between different set up, we are using results from Miller-Tans method with individual background subtracted and York fitting method, and treatment 1 averaging for bag samples or raw data for AirCore samples as the reference analytical strategy.”, which is used through the manuscript. We decided to use IRMS results instead of direct results as a reference value for indirect results, as we observe significant discrepancy between direct sampling and IRMS results. Thus, comparison only between indirect methods using IRMS reference value allows for better emphasis of observed biases introduced by different analytical strategies.
- line 311 ff: this paragraph is an interpretation and should not be part of the results section
A: This part has been removed from results.
- line 316-321: here, a figure would be helpful to emphasize the results.
A: This paragraph has been moved to another section, focused on averaging, to keep the order from flow chart in Figure 1. Figure 4 has been added for visualisation of the results.
- line 321,330 ff: recommendations should be no part of the results section.
A: This part has been removed from results.
- line 339 ff.: this is again an interpretation
A: This part has been removed from results.
- line 351-354: this information is redundant, as it can already be found in subchapter 2.2.
A: This part has been removed from results.
- line 368-369: I do not believe that temperature effects play such a large role in the observed isotope discrepancies. Since direct and indirect measurements are based on completely different conditions (differences in fluxes, flows, mixing effects), I would not have expected unbiased values (deviation from the ‘true’ direct measurements) for the indirectly measured isotope values.
A: We have tried to reword the main text in reply to the reviewer’s concern: lines 522-530 “We observed about a 1.1 ‰ discrepancy between indirectly (IRMS results) and directly determined δ13CH4 source. The observed discrepancy could be caused by different conditions in which direct and indirect samples are collected. For indirect studies, the gas was released over 45 minutes from the cylinder at high rates (up to 70 l min-1). For direct sampling, gas was released from one cylinder to another in less than two minutes. During the controlled release experiment we did not observe significant differences between δ13CH4 source indirectly determined during different releases when CH4 flow rate, wind speed, or wind direction varied (Tab. A1). These physical phenomena alone only impact the dispersion characteristic of CH4 from the source to the sampling location, without any plausible mechanism for isotopic fractionation within the boundary layer on these short timescales. Further studies on possible isotopic fractionation during gas release are planned in the future to verify observed discrepancy.”
Discussion
- line 384: The analytical technique used depends primarily on the environmental conditions and requirements, and on the sensitivity required (relative or absolute measurements). For field measurements, it is not always possible to take bag samples and store them for IRMS measurements.
A: Here, we wanted to highlight that if bags are collected, they should be measured on IRMS instead of CRDS and that using AirCore with CRDS gives better results than measuring bags on CRDS in the laboratory. We have rewritten the sentence to clarify the conclusion: lines 451-453 “Due to the bigger sensitivity to applied analytical strategy and higher uncertainty, comparing to IRMS results, we do not recommend measuring bag samples using CRDS. To use CRDS instrument for determination of δ13CH4S, AirCore tool should be implemented for in-situ CRDS sampling, as this measurement techniques gives more robust results.”
- line 388: The Miller-Tans-method should be used in any case where the background varies (usually in long-term studies). If the background is constant or not measured, the Keeling method can be used. Accordingly, there is not really a question which of the two methods is the better one, because the application depends on measurement conditions.
A: There are not clear thresholds to define when one method is preferred over another; there is a continuum of conditions over which either method could be applied. Our mobile measurements describe such complexity where we observe also a short term change of background CH4 mole fraction. In the morning, after night-time accumulation, CH4 mole fraction can be significantly higher than the daily average and decrease with time, especially during winter months. Moreover, δ13CH4 of sources are compared between each other in different locations and to observe possible changes over time (e.g., seasons, years). Thus, Miller-Tans method is more suitable for mobile measurements even though there are short-term studies of one individual source. As some groups use or Miller-Tans or Keeling method, we compared results of both of them to see if any method implement bias and can affect comparison of results from different groups.
- line 390: Hoheisel et al. (2019) seem to come to different conclusions: “Especially for natural gas samples, the precise determination and correction of C2H6 is important as in our study C2H6 can bias 13CH4 by up to 3‰ depending on the CH4-to-C2H6 ratio of the sample and the calibration cylinder.” How can you explain this discrepancy?
A: During the control release experiment, we were using exactly the same instrument as Assan et al. (2017) used to define and test ethane correction. Tests conducted by Assan et al. (2017), were repeated and results are in good agreement with previous studies (Defratyka, chapter 2 and 3, 2021). The same methods were used by Hoheisel et al. (2019) giving similar results. Overall, applying ethane correction, the δ13CH4 is shifted toward more depleted values as expected from previous studies (Rella et al., 2015; Assan et al., 2017; Hoheisel et al., 2019). However, the previous studies did not compare obtained CRDS results with IRMS studies or with direct sampling, what we do here. Potentially, it opens room for the discussion if ethane correction should be used or not. We have mentioned it in revised version of the manuscript: lines 465-469 “According to our knowledge, this is the first study, when CRDS AirCore results are directly compared to δ13CH4 source determined from bag samples measured on IRMS and direct source sampling, and providing evidence to exclude an C2H6 correction for measuring δ13CH4 in these types of samples. Previous studies implementing an C2H6 correction focused only on CRDS AirCore studies without the comparison with independent C2H6-interference free measurements. (Rella et al., 2015; Assan et al., 2017; Lopez et al., 2017; Hoheisel et al., 2019).”
- line 400-402: there are some more studies for example Takriti et al. (‘Mobile methane measurements: Effects of instrument specifications on data interpretation, reproducibility, and isotopic precision’, 2021) and Hoheisel et al. (2019). So why are you focusing only on the studies mentioned in line 401-402 to compare your results?
A: In the discussion comparison with Hoheisel et al. (2019) has been added: 489-501 “Hoheisel et al. (2019) focused on comparison of Keeling and Miller-Tans methods, using OLS and York fitting, for CH4 obs AirCore and synthetic data. Regarding comparison of measurement techniques, they obtained identical results for Keeling and Miller-Tans methods using York fitting. Using OLS fitting, they observed differences from -2 ‰ to 2 ‰ for individual AirCore samples between Keeling and Miller-Tans methods. Hoheisel et al. (2019) showed that in the case of AirCore studies, results of York fitting lie between results from OLS using Keeling or Miller-Tans method for 90% of measurements. In their study, in the case of synthetic data, results from York and OLS are nearly the same, having larger differences to true, modelled value. The observed discrepancy between fitted and true value achieved <0.2 ‰. These results are consistent with our study, where we did not observe significant differences between York and OLS fitting and bias between indirect and direct results achieved 0.3 ‰ for AirCore samples. Additionally, Hoheisel et al. (2019) checked influence of averaging time up to 1 minute. Using synthetic data, they demonstrated no significant differences between raw and 15 s averaged. Also, they observed the improvement of precision of the measurement during averaging over 1 minute, which did not improve δ13CH4 source signature determination. This is in opposition to our results from mobile measurements, where we observe the varying bias introduced by data averaging, which worsen determination on final δ13CH4 source. ”
We decided to not include comparison with Takriti et al., (2021), as their study focus on instrument performance tests, especially regarding response time. Some of discussed issues were previously tested in details by Hoheisel et al. (2019). Additionally, used in our studies CRDS had higher than default flow rate (100 ml/min versus typical 25 ml/min) to increase response time. Moreover, for in-situ sampling we used AirCore tool. Thus, increased flow rate and using AirCore tool makes difficult comparing between our set up and setup presented by Takriti et al., (2021).
- line 415: not only Wehr and Saleska proposed York fitting method, but also many other studies come to the same conclusion such as for example Hoheisel et al. (2019)
A: We have also included conclusion of Hoheisel et al. (2019) in the revised version (lines 479-491).
- line 420-422: Wehr and Saleska performed model simulations within their study. It is difficult to compare this result with the results of a field test conducted as part of this study.
A: Based on our understanding of the work of Wehr and Saleska, they created synthetic data as a representation of real-life dataset and using Monte Carlo simulation they compare results for York, OLS and GMR fitting. Here, we use real-life set of data to observe impact of different analytical strategies for determined δ13CH4. Thus, our case study are validation and implementation of results from synthetic data studies to real-life measurements.
- line 424-427: what is the conclusion from that statement? Instrument settings have not been discussed before, and introducing another parameter at this point is inappropriate. Therefore, I see no benefit in mentioning this here.
A: Thank you for highlighting this. We have removed this part from the revised manuscript.
- line 428-429: this is a self-evident fact that applies to all occasions that require sample dilution.
A: This part has been removed from revised version.
- line 430: I would rather speak of expectable then remarkable (see last comments on results). In general, possible fractionation effects should be checked before starting the actual experiment: e.g. due to different wind speeds, different inflow angles, different temperatures etc. As long as the experimental framework conditions are not clarified, an experiment that investigates numerous parameters is not useful, in my opinion.
A: Isotopic fractionation at the point of release is the only plausible way that we understand could affect isotopic composition during transport from the primary source to sample location. In the revised version we have added paragraph about observed discrepancies: lines 522-530 “We observed about a 1.1 ‰ discrepancy between indirectly (IRMS results) and directly determined δ13CH4 source. The observed discrepancy could be caused by different conditions in which direct and indirect samples are collected. For indirect studies, the gas was released over 45 minutes from the cylinder at high rates (up to 70 l min-1). For direct sampling, gas was released from one cylinder to another in less than two minutes. During the controlled release experiment we did not observe significant differences between δ13CH4 source indirectly determined during different releases when CH4 flow rate, wind speed, or wind direction varied (Tab. A1). These physical phenomena alone only impact the dispersion characteristic of CH4 from the source to the sampling location, without any plausible mechanism for isotopic fractionation within the boundary layer on these short timescales. Further studies on possible isotopic fractionation during gas release are planned in the future to verify observed discrepancy.”
- line 436-439: I consider this statement questionable as the experimental conditions are very specific (artificial release of gases to the atmosphere).
A: The control release experiment is a good representation of industrial settings and real life the conditions when mobile measurements of methane sources are made. Thus, we conduct the analysis of available analytical methods implied to real life data, when we can observe true impact of instrument (frequency, precision), methane range and meteorological conditions. Additionally, we have rewritten the manuscript to “a case study” instead of “general recommendation”.
Conclusion
- line 447: I would not speak of statistical methods, but rather of a comparison under different methodological and analytical aspects
A: The sentence has been rewritten.
Appendix
- Appendix B should appear in a shortened version in the main part of the manuscript (one to two sentences)
A: In the revised version, the appendix B has been shortly explained in main part of the manuscript.
- At the end of Appendix B a title (‘Results for bag samples measured on IRMS and CRDS for all examined linear fitting methods’) appears without text following
A: This part of Appendix B contains only figures and tables which were added at the end of the manuscript. In the revised manuscript, Appendix tables and figures are moved into appendix part of the manuscript.
- Appendix A: A figure would help to clarify the impact of data clustering. I would include this section in the main part of the manuscript.
A: The main findings of clustering impact have been added in the main text and figure 4 has been added.
- for the mobile sampling laboratories, a table with specifications would be beneficial for a better overview
A: A table A2 with specification of mobile laboratories and used instruments has been added in Appendix A.
Figures and Tables
- Figure 1: The figure is very clear and helpful to the reader. However, I miss the applied mobile laboratory and the measured components (CH4, δ13CH4) as a note in the graph
- Figure 1: inconsistency: the averaged AirCore samples are presented in ppb- in the main part they are given in nmol/mol
A: Figure 1 is improved based on reviewer’s comment.
- Figure 3: plotting different CH4 mole fractions based on the size of the data points is absolutely confusing - rather use an average value. A general advice: do not show more than one dependency in one figure
A: Figure has improved in revised manuscript.
- Figure 3: what does ‘release’ mean? Does this refer to the day of release or to the run number?
A: Here the releases are numbered in following order X.Y, where X stands for day and Y for the release made during individual day (e.g. 1.2 stands for the second release of the first day of the experiment).
- Figure 3: Do you really want to include the data from release 4-5, which is, as you mentioned in the text, biased by calibration errors? Since you also omit the data for averaging I would not show it at all as it confuses the reader.
A: In the revised version, the observed discrepancy is mentioned but data from day 4-5 are still included, as observed values are in agreement within uncertainty and there passed through rejection criteria. We will add in revised version: lines 514-519 “We observed that individual AirCore values for samples collected on days 4 and 5 of the controlled release experiment are more depleted than samples collected in days 2 and 3 (Fig. 2), but δ13CH4 source results are still in agreement within uncertainty. It is possible, that an unnoticed problem occurred with the instrument calibration or encountered mobile set-up leaks during those days. Based on this, we recommend measuring the calibration gases on each measurement day, both before and after the fieldwork.
- Figure B1, C1: these figures are not mentioned anywhere in the text!
A: These figures have been removed from the revised manuscript.
- Figure C1: use similar y-axis for better comparison
A: This figure has been removed from the revised manuscript.
- Table 3: probably transposed numbers for MA- δ13CH4 Keeling method? (-42.18 instead of -24.18?)
A: This value was double checked and there was no transposed number here. As explained in the text MA fitting introduce significant bias and value -24.18 ‰ is what we received from the calculation.
A: All proposed corrections for Figures and Tables have been implemented in the revised version.
technical corrections:
- please check again for correct super/subscript (i.e., CH4 instead of CH4) throughout the whole manuscript as there are many discrepancies
- line 130 ff: use consistently the term CH4s for source values to distinguish source values from observed values (see formula 1)
- line 31: delete the word ‘measurements’
- line 46: better write ‘equipped with an atmospheric sampling system (AirCore)’ as you have not introduced the AirCore system yet
- line 51: ‘determine δ13C source values’
- line 146: delete commas in that sentence
- line 265-267: ensure that the formulas are displayed correctly: μmol∙mol-1
- line 268: space between treatmen t
- line 339-340: rewrite the sentence as it is not clear (3 times the word corrections!)
A: All proposed technical corrections have been implemented in the revised version.
Citation: https://doi.org/10.5194/egusphere-2023-1490-RC1
References
Assan, S., Baudic, A., Guemri, A., Ciais, P., Gros, V., and Vogel, F. R.: Characterization of interferences to in situ observations of d13CH4 and C2H6 when using a cavity ring-down spectrometer at industrial sites, Atmospheric Meas. Tech., 10, 2077–2091, https://doi.org/10.5194/amt-10-2077-2017, 2017.
Brand, W. A., Coplen, T. B., Vogl, J., Rosner, M., and Prohaska, T.: Assessment of international reference materials for isotope-ratio analysis (IUPAC Technical Report), Gruyter, 86, 425–467, https://doi.org/10.1515/pac-2013-1023, 2014.
Brownlow, R., Lowry, D., Fisher, R. E., France, J. L., Lanoisellé, M., White, B., Wooster, M. J., Zhang, T., and Nisbet, E. G.: Isotopic Ratios of Tropical Methane Emissions by Atmospheric Measurement: Tropical Methane δ13C Source Signatures, Glob. Biogeochem. Cycles, 31, 1408–1419, https://doi.org/10.1002/2017GB005689, 2017.
CCQM: Consultative Committee for Amount of substance; Metrology in Chemistry and Biology CCQM Working Group on Isotope Ratios (IRWG) Strategy for Rolling Programme Development (2021-2030), https://www.bipm.org/documents/20126/57465585/CCQM-IRWG+Strategy+document+2021-2030.pdf/41d93edc-c543-8ed4-883b-26e97ac93867, 2021.
Coplen, T. B.: Guidelines and recommended terms for expression of stable-isotope-ratio and gas-ratio measurement results: Guidelines and recommended terms for expressing stable isotope results, Rapid Commun. Mass Spectrom., 25, 2538–2560, https://doi.org/10.1002/rcm.5129, 2011.
Defratyka, S.: Characterizing methane (CH4) emissions in urban environments (Paris), Paris-Saclay, Gif sur Yvette, 185 pp., 2021.
NOAA/ESRL: https://gml.noaa.gov/ccgg/trends_ch4/.
Gardiner, T., Helmore, J., Innocenti, F., and Robinson, R.: Field Validation of Remote Sensing Methane Emission Measurements, Remote Sens., 9, 956, https://doi.org/10.3390/rs9090956, 2017.
Hoheisel, A. and Schmidt, M.: Six years of continuous carbon isotope composition measurements of methane in Heidelberg (Germany) − a study of source contributions and comparison to emission inventories, 2023.
Hoheisel, A., Yeman, C., Dinger, F., Eckhardt, H., and Schmidt, M.: An improved method for mobile characterisation of D13CH4 source signatures and its application in Germany, Atmospheric Meas. Tech., 12, 1123–1139, https://doi.org/10.5194/amt-12-1123-2019, 2019.
Lopez, M., Sherwood, O. A., Dlugokencky, E. J., Kessler, R., Giroux, L., and Worthy, D. E. J.: Isotopic signatures of anthropogenic CH4 sources in Alberta, Canada, Atmos. Environ., 164, 280–288, https://doi.org/10.1016/j.atmosenv.2017.06.021, 2017.
Maisch, C. A. W., Fisher, R. E., France, J. L., Lowry, D., Lanoisellé, M., Bell, T. G., Forster, G., Manning, A. J., Michel, S. E., Ramsden, A. E., Yang, M., and Nisbet, E. G.: Methane Source Attribution in the UK Using Multi‐Year Records of CH4 and δ13C, J. Geophys. Res., 2023.
Menoud, M., van der Veen, C., Lowry, D., Fernandez, J. M., Bakkaloglu, S., France, J. L., Fisher, R. E., Maazallahi, H., Stanisavljević, M., Nęcki, J., Vinkovic, K., Łakomiec, P., Rinne, J., Korbeń, P., Schmidt, M., Defratyka, S., Yver-Kwok, C., Andersen, T., Chen, H., and Röckmann, T.: New contributions of measurements in Europe to the global inventory of the stable isotopic composition of methane, Earth Syst. Sci. Data, 14, 4365–4386, https://doi.org/10.5194/essd-14-4365-2022, 2022.
Rella, C. W., Hoffnagle, J., He, Y., and Tajima, S.: Local- and regional-scale measurements of CH4, d13CH4, and C2H6; in the Uintah Basin using a mobile stable isotope analyzer, Atmospheric Meas. Tech., 8, 4539–4559, https://doi.org/10.5194/amt-8-4539-2015, 2015.
Takriti, M., Wynn, P. M., Elias, D. M. O., Ward, S. E., Oakley, S., and McNamra, N. P.: Mobile methane measurements: Effects of instrument specifications on data interpretation, reproducibility, and isotopic precision, Atmos. Environ., 246, https://doi.org/doi.org/10.1016/j.atmosenv.2020.118067, 2021.
Takriti, M., Ward, S. E., Wynn, P. M., and McNamra, N. P.: Isotopic characterisation and mobile detection of methane emissions in a heterogeneous UK landscape, Atmos. Environ., 305, https://doi.org/10.1016/j.atmosenv.2023.119774, 2023.
Citation: https://doi.org/10.5194/egusphere-2023-1490-AC2 -
AC3: 'Reply on RC1', Sara Defratyka, 18 Mar 2024
Publisher’s note: this comment is a copy of AC2 and its content was therefore removed.
Citation: https://doi.org/10.5194/egusphere-2023-1490-AC3 -
AC4: 'Reply on RC1', Sara Defratyka, 18 Mar 2024
Publisher’s note: this comment is a copy of AC2 and its content was therefore removed.
Citation: https://doi.org/10.5194/egusphere-2023-1490-AC4 -
AC5: 'Reply on RC1', Sara Defratyka, 18 Mar 2024
Publisher’s note: this comment is a copy of AC2 and its content was therefore removed.
Citation: https://doi.org/10.5194/egusphere-2023-1490-AC5
-
RC2: 'Comment on egusphere-2023-1490', Anonymous Referee #2, 13 Jan 2024
Comments on “Statistical evaluation of methane isotopic signatures determined during near-source measurements”
by Defratyka et al.,
The manuscript “Statistical evaluation of methane isotopic signatures determined during near-source measurements” compares different air sampling, measurement and fitting methods to determine δ13CH4 source signatures using Keeling and Miller Tans plots. During a five-day release experiment two mobile laboratories with different sampling and analytical methods were used to measure extensive data sets, which are combined with different data analysis strategy (averaging, linear fitting algorithm) to determine the best methods.
The structure of the manuscript is difficult to follow and large tables with similar values using different fitting algorithms do not contribute to clarity. The authors have not succeeded in focusing on the essentials and highlighting the novelties and value of this study for the scientific community. Results from individual tests, some of which were subject to large measurement uncertainties were used to make general statements about best practice. For such general statements, the collected data should be supplemented with theoretical calculations or synthetic data sets which, as previous studies have shown, are better suited for this purpose.Therefore, I cannot recommend this manuscript for publication in its present form
General:
General conclusions have been drawn and recommendations made based on insufficiently thorough measurement protocols that do not reflect the current state of the art. The CRDS analyzer was only calibrated in the laboratory before it was sent to the campaign and not at regular intervals during the campaign. This is not sufficient for accurate δ13CH4 measurements and is not state of the art for precise measurements. Other studies, e.g. Lopez et al. (2017), Assan et al., (2017) Hoheisel et al. (2019) calibrate similar instruments every other day, every day or twice a day during campaigns.
How does such a calibration strategy affect the results? What is the instrumental drift over time? How long is the time interval between instrument calibration and field measurement?In the conclusion part, CRDS measurements on sample bags are not recommended due to lack of precision. How long have these bags been measured? What was the precision? In the data files orders of magnitude of 3 permil were given. Again, other studies achieve better precision with similar instruments averaging over 20-30 minutes .
The paper Defratyka et al., 2021 (which is not cited in the manuscript, only the data set) used this release campaign to test the capability of the CRDS analyser to measure C2H6. There, the release experiment is well described and gives information about the lengths of each individual release period. Here, it was difficult to follow the actual CH4 release during the different days. In particular, when were the sample bags filled? When were aircores taken? Only in the data file was it possible to see that the 21 IRMS sample bag were filled over the 5 day period. A direct comparison of sample bags and aircore during one release was not presented. Instead a 5 day average was used for the comparison, which does not represent a realistic setup for real field measurements. The authors claim to fill the gap that sampling for isotope signature determination has never been tested with controlled release experiments. However, test of the best sampling strategy: such as how many bag samples or aircore measurements are needed etc. was not presented or evaluated.
The method for linear regressions can be better evaluated with modeled or synthetic data. This has been shown by Wehr and Saleska (2017) for CO2 or Hoheisel et al. (2019) for CH4. In this manuscript, the results of individual regressions on measured data were used to recommend the best approach. Controlling factors such as CH4 excess, difference between source and background 13C values, measurement uncertainties were not systematically investigated or related.
A large part of the paper is devoted to the comparison of the Keeling and Miller Tans methods. It is also quoted that “The Miller-Tans method can be useful to interpret studies, where the Keeling method assumption of stable background is unfulfilled or unknown, e.g. when studies are conducted over a long period of time (Lowry et al., 2020; Al-Shalan et al., 2022).”
The Miller-Tans method (Miller and Tans; 2003) was designed for "longer time series" with changing background conditions, such as multi-year time series and weekly flask samples. . If a year of these data is plotted, one needs to account for the changing background conditions. In case of a 45 min release, the Keeling plot would provide a better estimate of the source signature since we do not expect a changing background over the short period of a release. Most importantly, why would we use the global averaged CH4 mole fraction or a δ13CH4 value from Brownlow et al. (2017) that is one year prior to your experiment? These are unrealistic assumptions that do not improve the approach, but worsen it compared to the Keeling plot, or, if the assumptions happen to hit the right value, appear correct.
Citation: https://doi.org/10.5194/egusphere-2023-1490-RC2 -
AC1: 'Reply on RC2', Sara Defratyka, 18 Mar 2024
Answer to Referee 2 Comments on “Statistical evaluation of methane isotopic signatures determined during near-source measurements” by Defratyka et al.,
A: We would like to thank the reviewer for their valuable suggestions that led to an improvement of the manuscript. In the revised version, we have addressed the reviewer’s comments and have made significant changes to improve the manuscript structure, increase readability and highlight novelty of the study.
R: The manuscript “Statistical evaluation of methane isotopic signatures determined during near-source measurements” compares different air sampling, measurement and fitting methods to determine δ13CH4 source signatures using Keeling and Miller Tans plots. During a five-day release experiment two mobile laboratories with different sampling and analytical methods were used to measure extensive data sets, which are combined with different data analysis strategy (averaging, linear fitting algorithm) to determine the best methods.
The structure of the manuscript is difficult to follow and large tables with similar values using different fitting algorithms do not contribute to clarity. The authors have not succeeded in focusing on the essentials and highlighting the novelties and value of this study for the scientific community. Results from individual tests, some of which were subject to large measurement uncertainties were used to make general statements about best practice. For such general statements, the collected data should be supplemented with theoretical calculations or synthetic data sets which, as previous studies have shown, are better suited for this purpose.A: The purpose of the study was to implement the results of already existing theoretical calculations and synthetic data to real life case studies. The control release experiment is a good representation of the industrial settings and real-life conditions when mobile measurements of methane sources are made. Thus, we conducted an analysis of the available analytical methods applied to real life data, when we can observe the true impact of instrument (frequency, precision), methane range and meteorological conditions. Also, we compared δ13CH4 of the source determined using two different instruments (IRMS vs CRDS) which was not done before. Finally, regarding analytical methods, we focused on methods which were used in previous mobile measurements but not tested before (e.g. BCES linear fitting method, averaging vs clustering into methane range bins, background impact for Miller-Tans method). In the revised manuscript, we have improved the structure of the manuscript to increase the clarity and highlight the novelties of the study. The structure follows the steps presented in the flow chart in Fig. 1. The new figures are plotted to better deliver the message coming from the study and the observed discrepancies between the different analytical strategies. We have introduced a “reference analytical strategy” and the impact of the different analytical steps are presented compared to this reference analytical strategy.
Therefore, I cannot recommend this manuscript for publication in its present form
General:
R: General conclusions have been drawn and recommendations made based on insufficiently thorough measurement protocols that do not reflect the current state of the art. The CRDS analyzer was only calibrated in the laboratory before it was sent to the campaign and not at regular intervals during the campaign. This is not sufficient for accurate δ13CH4 measurements and is not state of the art for precise measurements. Other studies, e.g. Lopez et al. (2017), Assan et al., (2017) Hoheisel et al. (2019) calibrate similar instruments every other day, every day or twice a day during campaigns.How does such a calibration strategy affect the results? What is the instrumental drift over time? How long is the time interval between instrument calibration and field measurement?
A: This particular CRDS model was used during other campaigns like (Assan, 2017; Defratyka et al., 2021a, b). Before implementing the instrument during field campaigns, numerous laboratory tests were conducted to characterize the instrument performance (Defratyka, 2021, chapter 2). For example, the working gas was measured during 21 randomly chosen mobile campaigns during studies focused on Paris agglomeration. This test aimed to check the analyser stability and lack of influence of switching on/off analyser for CH4 and δ13CH4 values. The measured values were stable and did not change over time.
We note that not having the working gas to check the instrument stability is a setback of our studies. However, a three-point calibration of the instrument was made on a regular basis every few months, including one calibration just before the controlled release experiment. Differences between the obtained calibration parameters are statistically insignificant. Additionally to the regular three-point calibration, the working gas was measured just before and after the controlled experiment (one measurement on 06.09.2019 and two measurements on 14.09.2019) in the laboratory. The gas was measured over 20 minutes and after stabilisation last 15, minutes were measured. No significant drift was observed. Based on all mentioned laboratory tests, we are confident that the CH4 and δ13CH4 values are robust and the instrument was stable without significant drift during mobile measurements with AirCore and during bag analysis in the laboratory. In the revised part of the manuscript, we added a better explanation: lines 115-119 “Due to logistic restrictions, the cylinder with calibration or known working gas could not be used during the controlled release experiment. However, an additional working gas was measured in the laboratory day before and day after the experiment and no significant drift was observed. During independent measurement campaigns, additional working gas was measured on used here CRDS before and/or after randomly chosen surveys (Defratyka, 2021, chapter 2). Observed time drift was negligible and working gas measurements inside the vehicle did not improve instrument precision. Thus, 3-point calibration done before controlled release experiment gives robust results. Additionally, the instrument was tested in laboratory conditions to verify possible impact of external pressure and temperature for CRDS instrument. No significant dependence on the ambient pressure and temperature both for CH4 mole fraction and δ13CH4 was observed for CRDS instrument used during this study (Defratyka, 2021, chapter 2). ”
R: In the conclusion part, CRDS measurements on sample bags are not recommended due to lack of precision. How long have these bags been measured? What was the precision? In the data files orders of magnitude of 3 permil were given. Again, other studies achieve better precision with similar instruments averaging over 20-30 minutes .
A: During release experiment, ambient air samples were collected in 5 L bags, which allowed for about 20 minutes of measurements using CRDS G2201-i. After removing initial stabilization time, around 15 minutes of measurements were analysed. Standard deviation was used as uncertainty of both CH4 and δ13CH4. Hoheisel et al (2019) were measuring samples 15 minutes, analysing 10 minutes after instrument stabilization. Thus, shorter time than in our study. They used Allan standard deviation to determine uncertainty of CH4 and δ13CH4, what could potentially explain observed lower uncertainty in their case. Other possible reason why other studies achieve better precision is the dependence of δ13CH4 precision (calculated as standard deviation) from CH4 mole fraction. During previous laboratory experiments (Defratyka, 2021, chapter 2.2.2.2), δ13CH4 precision improved from 3,5‰ to 0,7‰ when CH4 mole fraction of measured cylinder increased from 1,96 ppm to 10 ppm. Similar tendency is observed for bag samples measured in this study. For bag with the lowest CH4 mole fraction (1.96 ppm) δ13CH4 precision achieved 3.5‰, while for bag with the biggest CH4 mole fraction (5.25 ppm), δ13CH4 precision achieved 1.3‰.
R: The paper Defratyka et al., 2021 (which is not cited in the manuscript, only the data set) used this release campaign to test the capability of the CRDS analyser to measure C2H6. There, the release experiment is well described and gives information about the lengths of each individual release period. Here, it was difficult to follow the actual CH4 release during the different days. In particular, when were the sample bags filled? When were aircores taken? Only in the data file was it possible to see that the 21 IRMS sample bag were filled over the 5 day period. A direct comparison of sample bags and aircore during one release was not presented. Instead a 5 day average was used for the comparison, which does not represent a realistic setup for real field measurements. The authors claim to fill the gap that sampling for isotope signature determination has never been tested with controlled release experiments. However, test of the best sampling strategy: such as how many bag samples or aircore measurements are needed etc. was not presented or evaluated.
A: In the revised version of the manuscript, the set-up of controlled release experiment has been better explained to fulfil all referee’s questions.: lines 79-93 “The experiment lasted over 5 days in September 2019 at Bedford Aerodrome, UK. Pure methane was released from a manifolded multi-cylinder pack, of eleven cylinders containing 999.6 ± 10.0 mmol mol-1, with initial pressure about 200 bars. The impurities in cylinders came from ethane (48 ± 10 µmol mol-1) and propane (0.149 ± 0.30 µmol mol-1). All 11 cylinders were filled at the same time from the same CH4 source, ensuring δ13CH4 source remained stable over the entire measurement period. The methane release rate varied between releases, up to 70 L min-1. During the release, CH4 was mixed with ethane (C2H6) in a varying ratio, giving C2H6:CH4 ratios from 0.00 to 0.07. The purity of the C2H6 was 999.9 ± 10.0 mmol mol-1, with impurities mostly from methane (2.27 ± 0.46 µmol mol-1) and propane (7.5 ± 1.5 µmol mol-1). To achieve required CH4 emission rate, CH4 was released simultaneously from all 11 cylinders, using the transportable flow control system (details in Gardiner et al., 2017)). Briefly, flow control system is designed and configured for the creation of ‘real-life’ gaseous emission scenarios, in a range of industrial settings. It allows for validation of methods for emissions monitoring applied during typical field conditions. The control system is built on six mass flow controllers (MFC), (Brooks Instrument, Hatfield, PA,USA). Four primary MFCs provide four independent emission sources, while two secondary MFCs allow for the introduction of purge and interferant gases into the primary flow. The flow control system is computer operated, allowing for the implementation of pre-written operational programs and the post-test analysis (Gardiner et al., 2017).” Additionally to averaged results (daily and 5 days average), the results from one individual release have been compared for CRDS and IRMS: 355-361: “Finally, the results are compared for individual releases where sampling using two different methods was done simultaneously (Tab. 1). For bag samples measured on IRMS and CRDS, simultaneous sampling happened during two releases of first day (release 1.1 and release 1.2). AirCore samples were taking simultaneously with bags for IRMS during third release over third day of the experiment (release 3.3). The residual between δ13CH4 source from bag samples measured on IRMS and CRDS achieved 0.66 ‰ for release 1.1 and 0.21 ‰ for release 1.2. For AirCore sampling residual is even smaller and achieved -0.09 ‰ for release 3.3. For both CRDS measurement techniques, for individual releases, residuals are smaller than uncertainty, and thus results from CRDS and IRMS instrument are in good agreement within the uncertainty (Tab. 1).”
Typically, choice of the best sampling strategy depends on measurements conditions, like source location (e.g. empty road with possibility to stop or traffic jam inside the city) and CH4 enhancement above background (i.e. for CRDS studies higher CH4 mole fraction translate into better precision of measured δ13CH4). Thus, it is difficult to draw general conclusion about the best sampling strategy as it depends on individual measurements campaign and usually it is a compromise between attempt to collect as much samples as possible, and surrounding conditions, including aspects like safety (e.g. stopping car for sampling can cause accident), location and meteorological conditions.
R: The method for linear regressions can be better evaluated with modeled or synthetic data. This has been shown by Wehr and Saleska (2017) for CO2 or Hoheisel et al. (2019) for CH4. In this manuscript, the results of individual regressions on measured data were used to recommend the best approach. Controlling factors such as CH4 excess, difference between source and background 13C values, measurement uncertainties were not systematically investigated or related.
A: Thanks to this comment, we remarked that the novelty of the study was not highlighted enough. Most of previous studies focused on CO2 or and Monte Carlo simulations (despite Hoheisel et al. focused on CH4 studies, also with using of AirCore). Here, controlled release experiment is used as representation of existing methane point sources in a range of typical industrial settings, as described in Gardiner et al. (2017). As studies are focused on mobile measurements, presented results are applicable to all mobile measurements using bag sampling or in situ sampling with AirCore, which are common techniques for δ13CH4 source determination. Here we focused on real life CH4 dataset, obtained from the controlled release experiment, which simulate real life industrial settings, with realistic range of CH4 enhancement, number of collected data points used for fitting line and real instrument uncertainty. Thus, we checked impact of real- life condition for studies of source δ13CH4. Also, we aim for testing all already used fitting approaches from previous studies of mobile CH4 source signature determination and check their potential impact on final result. Some of these methods were already tested, mostly in context of CO2 respiration. Also, we test here BCES method which is used for δ13CH4 source of methane but not tested as profoundly as other methods (OLS, ODR, York).
Also, during the release, methane and ethane were released with varying fluxes and wind speed and direction were measured (added Table A1 in Appendix). Overall, using different measurement methods, we did not observe significant variation of determined δ13CH4, depending on CH4 flow rate, ratio of C2H6: CH4 and wind speed and direction. We included this observation in revised version of manuscript.
Regarding relation between source and background, measurements uncertainties, and impact of number of measurements point, these studies were done previously by Hoheisel et al. (2019) and we were using their conclusion through our studies.
R: A large part of the paper is devoted to the comparison of the Keeling and Miller Tans methods. It is also quoted that “The Miller-Tans method can be useful to interpret studies, where the Keeling method assumption of stable background is unfulfilled or unknown, e.g. when studies are conducted over a long period of time (Lowry et al., 2020; Al-Shalan et al., 2022).”
The Miller-Tans method (Miller and Tans; 2003) was designed for "longer time series" with changing background conditions, such as multi-year time series and weekly flask samples. . If a year of these data is plotted, one needs to account for the changing background conditions. In case of a 45 min release, the Keeling plot would provide a better estimate of the source signature since we do not expect a changing background over the short period of a release. Most importantly, why would we use the global averaged CH4 mole fraction or a δ13CH4 value from Brownlow et al. (2017) that is one year prior to your experiment? These are unrealistic assumptions that do not improve the approach, but worsen it compared to the Keeling plot, or, if the assumptions happen to hit the right value, appear correct.
A: During mobile measurements we observe also a short term change of background CH4 mole fraction. In the morning, after night-time accumulation, CH4 mole fraction can be higher than usually and decrease with time, what was observed especially during winter’s months. Moreover, δ13CH4 of sources are compared between each other in different locations and to observe possible changes over time (e.g., seasons, years). Thus, Miller-Tans method is more suitable for mobile measurements even though there are short-term studies of one individual source. As some groups use or Miller-Tans or Keeling method, we compared results of both of them to see if any method implement bias and can affect comparison of results from different groups.
We used global values of CH4 and δ13CH4 as an additional random background to see how differently chosen backgrounds can affect the results. Notably, during mobile measurements, researcher can face different conditions, when local enhancement of CH4 can be observed and they face the decision of choosing the background. Thus, we wanted to show how changes in background can directly affect determined δ13CH4 of methane source. In the revised version of the manuscript, we have improved the explication: lines 211-219 “Next, to verify the sensitivity of Miller-Tans method for a differently defined background, calculations for two randomly chosen backgrounds with lower CH4 bckg mole fraction and δ13CH4 bckg than during the experiment were conducted: “random I” and “random II” background. For random I background, CH4 bckg mole fraction is defined as an average global CH4 mole fraction observed in September 2019, equals to 1.8707 ± 0.0011 µmol mol-1 (NOAA/ESRL), while δ13CH4 bckg is defined using value from Brownlow et al., as -47.2 ± 0.2 ‰ (2017), to verify if slight difference in background can translate into significant differences in determined δ13CH4 source. For random II background, the CH4 bckg mole fraction is set up the same as the random I background, but the δ13CH4 bckg was randomly set to -42.7 ± 0.2 ‰ to significantly differ from other δ13CH4 bckg backgrounds to better test the sensitivity of Miller-Tans method to subtracted background.” We have also rewritten the manuscript and updated the figures to highlight differences between all used strategies not only to focus on comparison between Keeling and Miller-Tans methods.
Citation: https://doi.org/10.5194/egusphere-2023-1490-RC2
References:
Assan, S.: Towards improved source apportionment of anthropogenic methane sources, Paris-Saclay, Gif sur Yvette, 160 pp., 2017.
Brownlow, R., Lowry, D., Fisher, R. E., France, J. L., Lanoisellé, M., White, B., Wooster, M. J., Zhang, T., and Nisbet, E. G.: Isotopic Ratios of Tropical Methane Emissions by Atmospheric Measurement: Tropical Methane δ13C Source Signatures, Glob. Biogeochem. Cycles, 31, 1408–1419, https://doi.org/10.1002/2017GB005689, 2017.
Defratyka, S.: Characterizing methane (CH4) emissions in urban environments (Paris), Paris-Saclay, Gif sur Yvette, 185 pp., 2021.
Defratyka, S. M., Paris, J.-D., Yver-Kwok, C., Loeb, D., France, J., Helmore, J., Yarrow, N., Gros, V., and Bousquet, P.: Ethane measurement by Picarro CRDS G2201-i in laboratory and field conditions: potential and limitations, Atmospheric Meas. Tech., 14, 5049–5069, https://doi.org/10.5194/amt-14-5049-2021, 2021a.
Defratyka, S. M., Paris, J.-D., Yver-Kwok, C., Fernandez, J. M., Korben, P., and Bousquet, P.: Mapping Urban Methane Sources in Paris, France, Environ. Sci. Technol., 55, 8583–8591, https://doi.org/10.1021/acs.est.1c00859, 2021b.
NOAA/ESRL: https://gml.noaa.gov/ccgg/trends_ch4/.
Gardiner, T., Helmore, J., Innocenti, F., and Robinson, R.: Field Validation of Remote Sensing Methane Emission Measurements, Remote Sens., 9, 956, https://doi.org/10.3390/rs9090956, 2017.
Hoheisel, A., Yeman, C., Dinger, F., Eckhardt, H., and Schmidt, M.: An improved method for mobile characterisation of D13CH4 source signatures and its application in Germany, Atmospheric Meas. Tech., 12, 1123–1139, https://doi.org/10.5194/amt-12-1123-2019, 2019.
Citation: https://doi.org/10.5194/egusphere-2023-1490-AC1
-
AC1: 'Reply on RC2', Sara Defratyka, 18 Mar 2024
Data sets
Dataset: Statistical evaluation of methane isotopic signatures determined during near-source measurements Sara M. Defratyka https://data.mendeley.com/datasets/vfbbdvp9w2/1
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 641 | 230 | 48 | 919 | 46 | 37 |
- HTML: 641
- PDF: 230
- XML: 48
- Total: 919
- BibTeX: 46
- EndNote: 37
Viewed (geographical distribution)
| Country | # | Views | % |
|---|---|---|---|
| United States of America | 1 | 361 | 39 |
| Germany | 2 | 101 | 11 |
| United Kingdom | 3 | 78 | 8 |
| China | 4 | 58 | 6 |
| France | 5 | 46 | 5 |
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
- 361