A flight-first approach for contrail detection using ground-based cameras
Abstract. Validating forecasts and models used in contrail mitigation efforts requires high fidelity observations. The ability to match individual contrails with their progenitor flights in such observations is vital. High-resolution ground cameras can offer such high-fidelity flight attributions through direct observations of contrail formation, something that low-resolution observation methods (e.g., geostationary satellites) cannot due to their optical limitations. Previous "detection-first" approaches using ground cameras have employed flight matching after detecting unattributed contrails and can thus still struggle with the attribution task due to having no inherent mechanism to distinguish between newer and older contrails, particularly suffering when in-frame contrail densities are large. Here, we present a "flight-first" approach that focally searches for contrails within dynamic bounding boxes around wind-advected flightpaths, enabling inherent flight matching for young contrails while reducing computational costs. We demonstrate our approach using ground cameras from the Global Meteor Network (GMN), the SAM2 asemantic segmentation model, and a simple random forest classifier model (RFM). In a case study of 3,506 flights across three weeks in the Southwestern USA, we detect 265 contrails with a precision of 0.99 and a recall of 0.74. The flight-first approach uniquely allows us to pinpoint where detections were missed by the SAM2-RFM pipeline, identifying cloud obscuration as a prominent driver of failure. We thus propose that our off-the-shelf approach can be deployed at-scale as a primary low-cost layer in a wider contrail detection strategy for model evaluation, serving to identify places and times for follow-up observations with more advanced observation techniques. The GMN is well suited to such a task given its global coverage and standardised components.
The authors introduce a new methodology for contrail detection and attribution in ground camera imagery, called "flight-first" detection. The method involves starting with advected ADS-B data and prompting an unmodified SAM2 segmentation network where it should look for contrail objects, followed by post-processing the segmentation mask statistics and feeding these statistics into a random forest classifier (trained on human labels of contrail detections from the same imagery). Training and analysis are performed on a training/test split within a 16-day case study of a single camera in the Global Meteor Network (GMN).Â
The idea has merit and could lead to scientific significance by applying it; however the poor scientific quality and presentation quality as the manuscript currently stands does not warrant publication without at minimum major revisions.
The poor presentation quality (in particular, frequent overly broad/unsubstantiated claims and imprecise language) hinders the ability to assess the scientific quality. I've noted lines where this happens below, but likely was not able to capture all. The authors can achieve improved scientific significance in a number of ways, possibly through developing the methodology further until it is (clearly and quantitatively) a state of the art method with notable advantages and/or derived insights unavailable to current detection-first approaches. Alternatively the paper might be reframed as partly/mostly a "position paper" identifying a promising new area of research (at which point I would defer to the editor who will need to determine if that is sufficient grounds for publication in AMT), or perhaps the authors could submit to a more suitable venue for a work-in-progress, such as a conference workshop.
* Overly broad / unsubstantiated claims
- Page 1, Line 14 ("(e.g., geostationary satellites) cannot due to their optical limitations."): overly broad claim; MTG shortwave 500m pixels maybe can but hasn't been reported in the literature yet
- Page 1, Lines 25-26 ("global coverage"): "global coverage" is an overly broad claim; lots of terrestrial airspace is not covered by GMN and certainly nothing over water
- Page 1, Lines 34-35 ("(2021).With the future growth of the aviation sector, this contrail climate impact will only grow,"): this language implies you know the future with high certainty; soften phrasing to match supporting citation (eg "if trends continue"...)
- Page 7, Lines 210-211 ("This lack of pretraining reduces the risk of bias from a hand-labelled dataset and thus increases generalis..."): this claim should be supported by data or else removed
- Page 12, Line 372 ("for flight-first approaches."): also for detection-first approaches, no?
- Page 17, Lines 470-472 ("The advantage of the flight- first approach is that we are able to pinpoint the underlying causes for contr..."): geostationary "detection first" approaches have included flight path information in the imagery that is presented to labelers (eg McCloskey et al 2021 and Ng et al 2023) and presumably those authors have the ability to use such flight path information to similarly diagnose underlying causes for contrails being missed...how does flight-first approach distinguish itself in this regard?
- Page 20, Line 525 ("where these are either post-processing steps or assumptions. We"): this manuscript details a large number of post-processing steps and assumptions in handling the SAM2 masks; what evidence is there that these post-processing steps and assumptions are a worthwhile tradeoff to make compared with the post-processing and assumptions being used for flight attribution in detection-first approaches?
- Page 21, Lines 549-550 ("This indicates an overidentification of contrail formation by the meteorological product."): the evidence in this manuscript doesn't warrant this claim, because the manuscript does not present any analysis that would distinguish the cause of the 17.1% precision being due to the meteorological product vs due to the observability constraints of the GMN camera system or limits of the human labeling or algorithmic labeling accuracies
- Page 22, Line 565 ("This indicates an overidentification of persistence by the meteorological product."): again this claim is not justified by the evidence in this manuscript
* Methodology
- Page 3, Lines 118-119 ("when the evaluation dataset is curated to eliminate cases where the asemantic segmentation model extracts..."): this type of post-hoc curation sounds very likely to inflate the metrics while not being reliably reproducible by external researchers
- Page 3, Lines 115-116 ("The aim of this work is not to propose a definitive implementation of the flight-first approach, but rather..."): typically this sort of framing would be accompanied by open source code to facilitate this modularity, why is the code only available upon request instead of being open sourced?
- Page 4, Figure 1 ("Figure 1: Two example frames from a GMN camera in the central-western United States, demonstrating the (a) ..."): diagonal contrails result in larger SAM2 prompt rectangles than are used in horizontal or vertical contrails, how much does the accuracy of this method depend on the angle of the contrail in the camera view? ...on the density of contrails inside the box? does the method specify which pair of opposing corners of the box are expected to contain the contrail based on the flightpath or might the wrong pair of opposing corners having a contrail erroneously attribute a contrail formation to the wrong flight path?
- Page 5, Figure 2 ("Figure 2: Projected field of view (FOV) for the GMN camera whose footage is shown. Note that distances are ..."): I'm surprised the shape of the FoV is a polygon with the GMN camera sited on the long side, I would have thought that the viewing angle would be expanding away from the camera location
- Page 6, Line 200 ("in accordance with the methods outlined in pycontrails"): "in accordance with" implies your own implementation, is that the case or was it done by pycontrails implementation?
- Page 8, Lines 250-254 ("We find overall, therefore, that sufficient guidance is required to aid the segmentation of the relevant co..."): if you're going to include these types of methodological comparisons in an appendix, the evidence/data used to justify these choices is an important part of the value to future researchers.
- Page 9, Lines 266-277 ("Small regions, defined as fewer than 20 pixels during the formation stage and 30 for a formed contrail, are..."): is sensitivity analysis provided on these parameter choices? on what data and metrics were the parameters empirically chosen? was a holdout set used to guard against overfitting? (some but not all of these details are provided later in the manuscript, and should be moved earlier)
- Page 11, Lines 322-323 ("flightpath closely, with only minor wind- and projection-related discrepancies, whereas a larger distance s..."): was pycontrails modeling of sedimentation or downwash considered as a means of reducing wind error? why/why-not?
- Page 12, Lines 354-355 ("While the skewed train-test split may introduce a risk of overfitting,"): more details about the train/test split are needed. Is it a chronological split or random? I see there are some details given much later in the paper but they should be moved earlier
- Page 13, Lines 406-407 ("On the other hand, too high of a threshold unnecessarily prolongs tracking and artificially extends the est..."): sensitivity analysis or metrics assessed against some groundtruth generally accompany this type of discussion
- Page 17, Lines 487-488 ("We note that a small number of flights lacked ADS-B data entirely, mostly military flights, or showed discr..."): how were the authors aware of military flights if not via ADS-B? what fraction of flights in the study were flights without ADS-B? what types of discrepancies caused flights to be excluded?
- Page 23, Lines 593-595 ("The most influential classification features in determining whether segmented features are genuine contrail..."): how was it determined these were the most influential?
- Page 25, Line 657 ("requested via the GMN."): contact information is probably needed to make this a realistic opportunity to obtain the data
- Page 27, Lines 733-735 ("Marc Shapiro, Zeb Engberg, Roger Teoh, Marc Stettler, & Tom Dean. (2023). pycontrails: Python library for m..."): which version of pycontrails did you use? I doubt it was v47 that was a while ago
* Missing definitions or quantifications
- Page 1, Line 21 ("USA, we detect 265 contrails with a precision of 0.99 and a recall of 0.74."): against what ground truth? (later in the manuscript IIUC it's specified as human labels, but at this point in the manuscript it is not clear)
- Page 3, Lines 120-121 ("persistent contrails with very high precision"): typical definition of "persistent contrail" in geostationary context is >= 10 minutes lifespan, is that what's also used here? (define at first mention)
- Page 5, Line 149 ("ultra short-lived contrails,"): define duration of this category at first mention
- Page 6, Line 169 ("a 350-pixel padding,"): unclear: how is 350-pixel padding applied if it's outside the field of view of the camera pixels?
- Page 10, Lines 320-321 ("The distance between the mask centreline and the flightpath, also computed with a distance transform, asses..."): distance in pixel space or in meters?
- Page 11, Figure 5 ("Flights without valid masks are automatically labelled as negative and are excluded from training and testi..."): it's not clear where the reader should look for details about what constitutes a "valid mask" that triggers exclusion from the metrics, to confirm whether that's justified or not
- Page 14, Lines 409-411 ("We categorise detected and tracked contrails into three classes, in keeping with Low et al. (2024). We defi..."): "ultra short" still not defined (nor analyzed at any point?)
- Page 15, Lines 440-441 ("and other cases."): what other cases?
- Page 15, Line 440 ("without meaningful segmentation output,"): what constitutes "meaningful" output?
- Page 17, Lines 484-485 ("However, excessive padding may lead to unrelated features being segmented before the contrail advects into ..."): how often?
- Page 19, Table 1 ("See footnote for definition of MCC."): footnote describes but does not define what MCC is or how it is calculated
- Page 20, Lines 512-515 ("Out of the two poorer contrail-observability days, one day yielded only true negative detections, while thr..."): language here should be clearer that this is with respect to human labeler (who was the human labeler? examples of the type of labeling setup they used and similarities/difference comparison to existing labeled contrails datasets in the literature is warranted, and citation if your labeling setup is following a particular other labeling scheme already published in the literature) ...so these quantifications are not about observability of contrails in GMN cameras under these conditions; this analysis is about how well the model does compared to the human labeler (eg some very far away contrails may not be visible in GMN cameras even to humans)
- Page 22, Lines 571-572 ("If a contrail’s persistence can be reliably predicted from its formation visual characteristics, paired wit..."): what type of visual characteristics and why might someone expect those visual characteristics to be predictive of contrail lifetime radiative impact?
- Page 22, Line 578 ("NWPs"): define at first mention
* Precision of language
- Page 1, Lines 31-34 ("with the global annual mean contrail radiative forcing estimated between 34.8 and 74.8 mWm-2 in 2019 (Teoh ..."): cites two different types of radiative forcing numbers without making clear the distinction between them (EF vs ERF)
- Page 1, Lines 37-38 ("The lifetimes of contrails are relatively short (e.g., Gierens & Vazquez-Navarro (2017) found an average of..."): clarification needed: this number is only for linear detectable contrails, not for non-linear contrail cirrus clouds
- Page 2, Line 56 ("deviations,"): 'deviation' is a term of art in the airline industry that means the aircraft went to a destination airport that wasn't its original destination; that is not what's happening with contrail avoidance adjustments
- Page 2, Line 67 ("reasonable accuracy"): avoid subjective and low-information phrases like "reasonable" unless you literally mean "has a reason"
- Page 2, Line 71 ("Euchenhofer et al. 2025),"): note that this paper's analysis is not actually able to make claims about geostationary detectability of a given contrail in *at least one* stage of its lifespan, rather it is making claims about the effect of spatial resolution on geostationary detectability when no temporal information / flightmatching is taken into account. please do not propagate undue alarmism about the utility of geostationary sensors for contrail climate analysis.
- Page 7, Lines 205-206 ("apart from instancing multiple objects."): what "instancing multiple objects" means is not necessarily going to be clear to the general readership at AMT
- Page 10, Lines 306-307 ("Our final operation thus involves removing persistent noise by labelling the mask volume into distinct clus..."): an individual 3d pixel is called a voxel; this parenthetical implies that a cluster of 3D pixels is the definition of voxels, which is incorrect
- Page 19, Table 1 ("Unseen Subset"): "RFM-excluded subset" would be a clearer name, if I'm understanding correctly
- Page 21, Line 534 ("ECMWF ERA5 HRES"): HRES (forecast) and ERA5 (re-analysis) are different products from ECMWF, which was used in this paper?
- Page 22, Line 566 ("ERA5 forecast"): ERA5 is not a forecast
- Page 22, Line 566 ("31km"): the spatial resolution of ERA5 is typically 0.25 degree and so the number of kilometers varies based on latitude
- Page 23, Line 596 ("advected traces,"): "advected trace" as a term in this discipine was introduced in Chevallier et al 2023, but does not mean what you are using it to mean here
* Citations
- Page 2, Line 41 ("it would provide the world with better living conditions"): citation needed; top of atmosphere radiative forcing are in my opinion the most well-constrained of contrail climate impacts and they are not generally considered "better living conditions"
- Page 2, Line 63 ("Contrail detection efforts to date have largely been based on imagery from geostationary satellites, such a..."): there's lots of LEO satellite literature also, should not be ignored in overview since it is one of the closest matches for spatial resolution to what the ground cameras offer.
- Page 3, Line 102 ("dry advection."): citation+summary needed for what 'dry advection' means
- Page 6, Lines 162-163 ("(Shapiro et al., 2023)."): this reference does not contain details about how to acquire Spire ADS-B data; AFAICT the spire.py file in pycontrails is for postprocessing the ADS-B data once you already have it
- Page 21, Lines 548-551 ("Formation recall is strong, with 92.8% of contrails forming in SAC-positive regions. However, formation pre..."): comparison should be made to precision and recall numbers reported in recent literature results with similar-resolution observations of contrail (Sentinel satellite and other ground camera papers for example).
- Page 22, Line 567 ("For instance, ISSRs were found to have a median pathlength of 3 km (Reutter et al., 2020),"): is this assertion actually supported by this citation? ISSRs are often hundreds of kilometers wide.
- Page 22, Lines 575-576 ("a phenomenon that is well-known within the contrail observation community."): citation needed
- Page 24, Lines 640-641 ("This is due to its high fidelity, from its inherent inclusion of negative detections as well as positive, a..."): evidence needed for claim about computational cost comparison to detection-first approaches. This may be one of the most promising potential differentiators of this method compared to detection-first approaches.
- Page 27, Line 733 ("Marc Shapiro, Zeb Engberg, Roger Teoh, Marc Stettler, & Tom Dean. (2023)."): correct formatting of this reference, it's Shapiro et al but alphabetically sorting under "Marc"
* Structure, typos
- Page 1, Line 33 ("4.0-3.4)"): should be low then high?
- Page 3, Lines 93-94 ("Such studies demonstrated the feasibility for ground-based flight-attributed contrail detection but also hi..."): should the Jarry et al eurocontrol ground camera dataset be cited here also? why is it in the next paragraph instead?
- Page 3, Line 120 ("is capable classifying persistent"): "capable of"
- Page 6, Line 164 ("flight ID’s,"): no apostrophe
- Page 7, Line 223 ("where the use of a video tracker could risk in propagating false positives,"): "could create risk" ?
- Page 7, Lines 234-235 ("Several potential pre-processing approaches were considered and discarded, which we outline here both to in..."): this sort of information belongs in an appendix (if anywhere), not the main methods section of the manuscript
- Page 9, Footnote 1 ("1 In one variant of the method, only pixels lying within the padded flight path envelope were kept, reliabl..."): strongly consider pruning "experiment log" narration from the final manuscript
- Page 12, Line 344 ("but it achieved inferior performance and hence was not chosen."): If you do want to include mentioning this, "(data not shown)" is typically added in these cases to prevent readers from wasting their time trying to hunt down the metrics elsewhere in the manuscript
- Page 12, Line 354 ("3,506.The model was trained with 90% of this data subset and tested with the remaining 10% (Fig. 5)."): "With test results shown in Fig 5" (as it currently reads, I expected to see more details about train/test split in Fig 5 but these details are not to be found.)
- Page 12, Lines 361-363 ("The RFM’s isolated evaluation thus reflects the performance achievable under the assumption that meaningful..."): providing a "ceiling" or "headroom" analysis to give evidence that a particular methodology can achieve a certain accuracy even if it's not achieving it yet can be valuable; however that sort of analysis must be made extremely clear as distinct from the current performance of the method. conflating the two in the paper by curating the dataset is a severe scientific issue with the manuscript as it currently stands.
- Page 14, Lines 427-435 ("Using our pipeline of input box construction, SAM2 segmentation, mask cleaning, metric calculation, RFM cla..."): these dataset construction details belong in methods section not in Results (probably first or early in the methods section given how many other places in the methods were alluding to these details without providing them)
- Page 17, Line 483 ("partially combatted is with"): "partially combatted with"
- Page 21, Line 528 ("Comparison to meteorology & modelling"): this paragraph topic belongs in a Discussions/Future-work section and doesn't fit in "Comparison to meteorology & modelling"
- Page 22, Line 568 ("ERA5s"): prefer "spatial resolution of ERA5"
- Page 26, Line 681 ("References"): bibliography formatting needs to be corrected, it's difficult to see where one reference starts and another ends