A Novel Segmentation Algorithm for the ARM User Facility All-Sky Imagers Using Machine Learning Applications

Silber, Israel; Flynn, Donna M.; Comstock, Jennifer M.; Cromwell, Erol L.; Ermold, Brian D.

doi:10.5194/egusphere-2025-6134

Preprints

https://doi.org/10.5194/egusphere-2025-6134

Preprints

23 Feb 2026

| 23 Feb 2026

A Novel Segmentation Algorithm for the ARM User Facility All-Sky Imagers Using Machine Learning Applications

Israel Silber, Donna M. Flynn, Jennifer M. Comstock, Erol L. Cromwell, and Brian D. Ermold

Abstract. Clouds play a pivotal role in modulating the Earth's energy budget through the reflection of incoming solar radiation and the trapping of outgoing longwave radiation. Ground-based all-sky imagers offer an objective assessment of cloud cover and can be used to estimate solar irradiance, classify cloud types, track cloud movement, and serve as a benchmark for the evaluation of satellite and reanalysis data products. The Atmospheric Radiation Measurement (ARM) user facility has utilized all-sky imagers for more than 25 years to monitor cloud cover and augment its comprehensive suite of atmospheric measurements. Following the retirement of its Total Sky Imager (TSI), ARM recently deployed the TSI’s successor, the All-Sky Imager (ASI-16 camera systems). To provide a smooth transition and continuity to the vast amount of knowledge gathered by the TSI over the years, while addressing typical deployment issues, we developed a novel pixel segmentation algorithm, the ASI Sky Cover (ASISKYCOVER). ASISKYCOVER builds on the different strengths and properties of the TSI processing algorithm while integrating machine learning techniques, ensuring data validity and accuracy across diverse atmospheric conditions. It enhances cloud cover characterization with new features such as artifact detection and uncertainty quantification. ASISKYCOVER also includes cloud cover estimates for near-zenith (narrow field-of-view) and reduces susceptibility to false detections. This study introduces ASISKYCOVER, details its algorithm framework, and demonstrates its capabilities using a year-long dataset from the ARM Southern Great Plains site. Comparisons with co-located TSI data and other ARM measurements, such as zenith-pointing radars and lidars, are presented, underscoring the ASISKYCOVER’s potential to improve cloud cover analyses and data evaluation efforts, as well as to be integrated into higher-level data products that synergize instrument suites to generate new and insightful information.

Received: 09 Dec 2025 – Discussion started: 23 Feb 2026

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors. Views expressed in the text are those of the authors and do not necessarily reflect the views of the publisher.

Download & links

Israel Silber, Donna M. Flynn, Jennifer M. Comstock, Erol L. Cromwell, and Brian D. Ermold

Status: final response (author comments only)

RC1:
'Comment on egusphere-2025-6134', Anonymous Referee #2, 16 Mar 2026
This paper introduces an approach to performing cloud cover analysis by means of image segmentation, designed specifically to work with a new type of all-sky camera operated by the Atmospheric Radiation Measurement (ARM) program, which replaces an older camera model used for multiple decades. The objective of the algorithm is to provide continuity to cloud cover estimates from all-sky camera images by the ARM program, and perform well at multiple sites of deployment.
The paper is well-written, and the visualizations are helpful (but image resolution needs to be improved for some of the figures, see line-by-line comments). The algorithm’s performance is assessed both quantitatively (using a test dataset) and more qualitatively (by comparison with other collocated measurement data), and this performance seems fit for purpose.
My main critique of the current paper is that it reads more as an algorithm description document than a scientific paper. The scientific value of the algorithm and its output is clear to me, but I think the paper could improve in placing the algorithm in context of other approaches discussed in the literature and how the performance compares to those. Specifically, the method to classify cloudiness operates on a per-pixel basis, which is likely inferior to segmentation algorithms that consider the entire image at once (e.g. a convolutional neural network). Although it is mentioned that computational efficiency is required for generating the algorithm’s output in near real-time, it is not exactly clear what the limiting factors are and whether the simplified approach is justified: especially when little to no successfully implemented alternative approaches are mentioned. It is thus also difficult for the reader to assess the performance of the algorithm. As an example, the paper states that “Clear sky pixels are properly classified at an impressive rate of 91%”, but how can we possibly say that this is impressive if no reference performance is given?
Some other broader comments/questions I have:
The discussion of the RSS approach for the uncertainty quantification needs more detail, or at the very least a reference to a resource that further explains it.

The calibration of the uncertainty estimates for the cloud coverage is not assessed. Firstly, when the algorithm says that the cloud cover is 50% +/- 10%, what does this mean? If it means something like “there’s a X% chance that the actual cloud cover is between 40% and 60%”, then the correctness of this uncertainty band could be assessed quantitatively.

Given that bounding boxes are used for labeling, I imagine that very few areas near cloud boundaries (which are typically not rectangular) have been incorporated into the labeling process. How does this affect the algorithm’s performance? Shouldn’t you create some pixel-by-pixel annotations to really understand the performance of the algorithm?

One of the objectives of the algorithm is to perform well at different deployment sites. But this is not explicitly assessed in the discussion of the pixel segmentation algorithm, correct?

Line-by-line comments:
Figure 1:
Missing horizontal axis label for histogram.

Different colors for the class bounding boxes are occasionally hard to discern from each other. Perhaps add text labels to them within the images?

Generally, the figure resolution is a bit too low.

It would be good to add a title to each image that indicates clearly what deployment it is from and what the date + (local) time is. The information is there already in the filename, but it’s not very clear.

Line 102: should “exists” not be “exist” ?
Figure 2:
Would also benefit from a higher resolution.

Some of the uncertainty estimates would lead to physically infeasible cloud cover values (e.g. fig b has a lower uncertainty bound below 0%). It may be worth commenting on this, at the very least.

Line 162: “bogus cloud cover” is more exact terminology available to describe this?
Line 178: “mis-alignment” spelled as “misalignment” elsewhere in the paper
Equation 2: Might be helpful to state the units of the angles here?
Line 244: one symbol is italic, the other isn’t. And is “solar zenith angles” here correct?
Line 260: Shouldn’t all the numbers here be written as 75^th, 95^th etc?
Figure 5:
Quite hard to distinguish between the KAZR and CEIL curves.
Citation: https://doi.org/10.5194/egusphere-2025-6134-RC1
RC2:
'Comment on egusphere-2025-6134', Anonymous Referee #1, 25 Mar 2026
General Comments:
This manuscript presents a significantly updated and improved image segmentation approach developed for the All Sky Imager (ASI) deployed at Atmospheric Research Measurement (ARM) user facilities after the retirement of the long-running TSI instrument. The paper is written with clear prose and thorough analysis, and promises to be a highly cited paper, as every future investigation that uses the sky cover or cloud mask output will cite this paper. So it is a timely, relevant, and highly useful publication for the community. I provide ample comments below in my efforts to improve the clarity and relevance of the message in this paper, even though the paper is already of a very high quality and relevance. The comment platform restarts all numbering - see the attached PDF for sequential numbering.
Specific Comments:
I would like the paper to explicitly provide guidance on how future users of this dataset should use the “uncertainty” information. This comment is motivated by my observation that researchers may tend to ignore or incorrectly use uncertainty metrics. For example, in the discussion of Figure 3c (page 10), it was indicated that the 23.7% confusion of “masked” pixels being classified as “cloudy” pixels was attributed to the “halo” found around artifacts such as the bird, and these pixels are treated as “uncertain.” So … is there a way to exclude “uncertain” pixels from the final product (sky cover and cloud mask images)? What would be the benefits and drawbacks from simply excluding all “uncertain” pixels from the final products? (For example, in Figure 2, the cloud masks would convert all of the hatched areas into masked areas.)

A sentence or two about the approach to the ASI near-zenith product would be helpful. I was surprised to learn in the caption to Figure 8 that it requires 99% or more of (~5400) pixels within the 5 degrees off-zenith. This didn’t make sense to me. How many pixels fall in this 5-deg circle around zenith? Is the ~5400 pixels the product of the pixels in this circle times the number of images? I would assume that a 5-minute running mean was also taken, consistent with the processing for KAZR and CEIL?

Since this will probably be the canonical paper for utilization of the ASI data stream, have you considered including a photograph of the instrument? I noticed that one is not included on the ARM ASI instrument page, and it could be nice to see what it looks like! (just a suggestion – your choice.)

I would like to know more about how the manually determined training and testing pixels were created. I see now, after carefully reading the paper, that the labeling process is illustrated in Figure 1, but that was not apparent upon my first reading.

A) This could perhaps be improved by a first sentence in the caption to Figure 1 indicating the main purpose of the figure, before highlighting the distribution of training & testing pixels relative to SZA? Moreover, the histogram could come at the bottom of the figure, because it seems secondary to the labeling boxes in panels b-j. Or the histogram could get moved to a separate figure altogether, perhaps that includes other qualities of the sampled pixels.

B) I also see that the discussion about manual labelling is located in the paragraph around line 175. It seems odd to me that if “… we attempted to sample the various features in a uniform matter …” then only the distribution of SZA is included in Figure 1. I would expect to also see the distribution of the different classes (clear, thin, etc.), and the distribution of pixel location (elevation and azimuth angle).

C) (curiosity question) Does the output of this segmentation model appear to be sensitive or insensitive to the sampling and labeling of the manual dataset?

I appreciated the discussion about the calibration for the solar zenith and azimuth angle, but I feel that the prose should be more clear, and an equation might facilitate communication, particularly the paragraph around line 110. A) That is: The analytical SZA and azimuth is well known from astronomical factors. I understand that you are using a manual labeling approach to identify the center of the sun location on the image. What is a “denoted image”? When you mention “calculated angles,” is this the true SZA? Expected image location of the SZA? Manually determined SZA? Or difference between the expected image location and manually determined location? The same four questions hold for the azimuth angle, of course! I recommend clear and consistent language. A simple equation might help to clarify this discussion significantly. B) Furthermore, it is stated that “the positioning biases characterized for different ARM deployments are robust (e.g. see concentric yellow circles in Figure 1)” and indicates that the “denoted versus calculated angles” have a standard deviation on the order of 1 degree. I’m not sure what “robust” means here – small? Consisitent over time for each deployment? – but examination of Figure 1 shows that the yellow and green circles show significant disagreement in 1b, c, i, and j (4 out of 9), and are noticeably different in all other thumbnail images – e.g. the green circle is not overlapping the yellow circle. I would expect a 1-degree difference to be indistinguishable by eye, so that suggests that I don’t understand something fundamental about the “denoted versus calculated” angles. Please clarify!

Technical Corrections:
Line 51: There is a citation to (ASI-16) that I cannot find in the bibliography. Is there a publication that describes the fundamentals of the ASI deployed at the ARM sites, such as image size (pixel dimension), image capture frequency, dynamic range (presumably 256 bit), and anything else? If not, then this publication could serve this need!

Line 85-90. This indicates that the image center is found by the black circle in the ASI images. Yet the black circle must be set by the imaging rig, right? Because the world doesn’t become black below the horizon. So what makes us think that the center of this black circle represents true zenith?

Lines 100-105. I found myself wondering about possible barrel distortion here, and you discuss it around lines 135-140. Consider moving the lens distortion discussion to this section. (optional – your choice!)

Figure 1
The axis labels are too small to read, including the legend in 1a and the SZA & AZ annotations in each thumbnail.

The colors of the boxes are not at all clear from the text description in the caption (orange, yellow, purple and brown). Could you please include a legend with the colored lines, because I cannot tell them apart in the images!

See my comments above suggesting that the histogram comes at the bottom of the image, the caption begins with an orientation for the reader, and/or the histogram gets moved to a separate figure that contains histograms for other properties as well.

Even with a legend with line color, it is tremendously hard to see the boxes and colors. Could this image be enlarged to full-page size?

I am curious – where is “North” in these images? I would expect it towards the top of the image, with “East” towards the right, but the azimuth angles don’t support that assumption. Could you please add a compass rose or North arrow for clarification?

Figure 2
Again, the font size for the legends in the “cloud masks,” the colorbar labels, and the image titles are too small to read!

An image such as this begs for a “truth” column to the right of the segmented image to enable a discussion of the spatial patterns that have been accurately or inaccurately labeled. Which leads me to wonder: what would be the most perfect, “true” segmentation result for these four images? (!)

Table 1: is the third column needed?

Para near line 175: Perhaps consider adding the geographic context (e.g. Tasmania, Alabama, US) for the field sites that don’t include the geographic location, and also for the CuORAGE site around line 199. (suggestion)

A citation or additional information about the root-sum-square analysis would be helpful

Figure 3.
Avoid red font on a blue background! I suggest a grayscale background with black text on the non-diagonal cells, and white text on the diagonal cells.

Fix the massive size of (a), (b), and (c).

Line 244: theta_i1, and theta_i2 should not be the SOLAR zenith angles, but perhaps the pixel zenith angles? Also, the “vertical edges” of a pixel is not clear – do you really mean the y-axis limits (vertical)? Wouldn’t a conversion to radial coordinates make more sense? Otherwise with “vertical” pixel zenith angle for a pixel at y ~ 800 would be identical. (using the pixel counts from Figures 1 & 2)

Line 245. I am curious – if the N_valid does not include “uncertain” pixels, then presumably neither does N_cld. So I wonder then where the “uncertainty” comes from, for example in Figure 8, where it appears to be around 10-20%?

Line 282. Sentence beginning “In addition …” The previous sentence discussed the cloud cover around sunset, but this sentence appears to discuss the morning, Please provide a transition for the reader.

Line 285 and the thumbnail sky image in Figure 5 at 14:00: You cite “the challenge in even manually interpreting …” It is entirely unclear from the print copy of this paper what is going on in the thumbnail at 14:00! It appears to be a clear (blue) sky? Yet the ceilometer reads 100% cloud cover (< 7 km), and the text indicates low clouds. Could the experts please clarify what the “true” sky cover is like in this case, for example?

Figure 5:
Again I find myself wondering about north, since the sun appears to set in the southeast.

The text discusses SZA = 80-deg. Could you indicate when that occurs in the figure? (e.g. civil twilight etc., or time of day?)

It might be nice to include lines that connect each image thumbnail to the time of its capture, since they don’t all appear immediately under their image time. I think this would help you tell your story.

Any chance you could enlarge the font size for the axis tick labels and axis labels? (I understand that the legend size is limited!) That said – please reduce the font size for the thumbnail image times!

Consider adding the ASI-determined cloud cover (f) for each thumbnail.

Full year analysis: I am curious about times when the thin clouds (presumably higher altitude) might be occluded by the lower, thicker clouds. I’m not sure if this is relevant or important, though, given the snapshot-nature of this data presentation.

Full year analysis: a number of cloud climatologies have been done at the SGP site. Is there any way to compare this one-year output to any of those published datasets?

Figure 6 (all optional suggestions for your consideration)
You might consider a different color choice, so “thin” corresponds to a lighter color, and “thick” to a darker color for easier interpretation.

I would also recommend text labels under each shaded region reading, for example “thick,” “thin,” and “intermediate” for easier interpretation.

I’m sure you carefully designed the figure, but I would have expected the “thick” category to be on the bottom, partially because it is the primary ASI observation since it tends to occur due to lower clouds. Also, when “thick” is high, I would expect more “thin” clouds to be occluded, indicating that the shaded regions should be dependent upon the lower region.

Figure 7
Again, please reduce the font size for (a), (b) etc.

I find the color contrast in Figure 7b challenging, between the 25 < CC < 50 and the 50 < CC < 75 conditioning. Perhaps a different hue?

In figure 7c, I’m curious about the mostly clear images. Are there any images with CC = 0%? If they have one pixel that is “thin,” does it fall in the “thin dominated” category? I wonder if a lower limit on CC is appropriate here, to help account for clear sky images in which no cloud type is dominated?

Consider including in the caption the data analyzed: presumably the SGP images from 2024?

Line 372: “(or clear sky pixels as clear for that matter, see Figure 3)” This is not apparent to me! The clear classification (horizontal row – true label) looks good in image 3a!

Line 375: I feel that this final sentence of this paragraph is an important key conclusion of your paper! “the dominating cloud type per scene might be the best predictor for cloud detection algorithm consistency.” Consider highlighting this in the abstract & conclusion? Unfortunately, the second clause to this sentence is not clear to me (“and emphasize the model performance …”)

Line 381: regarding ASI solar flaring, perhaps refer to Figure 1, panels c-i?

Figure 8:
Does this figure again analyze the 2024 observations from SGP?

In the bar chart, does “Number of samples” correspond to images? Pixels? 5-minute observations?

I also want to make sure that the x-axis label for the bar chart should truly

Line 411 in the caption to Figure 8: close the parentheses after (see Section 3.1).

Line 447: “gamma configurations” is not clear.
Citation: https://doi.org/10.5194/egusphere-2025-6134-RC2

Israel Silber, Donna M. Flynn, Jennifer M. Comstock, Erol L. Cromwell, and Brian D. Ermold

Viewed

Total article views: 205 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
135	54	16	205	15	25

HTML: 135
PDF: 54
XML: 16
Total: 205
BibTeX: 15
EndNote: 25

Views and downloads (calculated since 23 Feb 2026)

Month	HTML	PDF	XML	Total
Feb 2026	54	22	6	82
Mar 2026	73	28	10	111
Apr 2026	8	4	0	12

Cumulative views and downloads (calculated since 23 Feb 2026)

Month	HTML	PDF	XML	Total
Feb 2026	54	22	6	82
Mar 2026	73	28	10	111
Apr 2026	8	4	0	12

Viewed (geographical distribution)

Total article views: 207 (including HTML, PDF, and XML) Thereof 207 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 06 Apr 2026

Short summary

We describe ASISKYCOVER, a new machine learning algorithm for pixel segmentation of all-sky imager (ASI-16) data used by the Atmospheric Radiation Measurement (ARM) User Facility. ASISKYCOVER provides cloud cover and thickness estimates, detects artifacts, and reports uncertainties. Using one year of data from the ARM Southern Great Plains site and comparisons with other ARM datasets, we demonstrate its use and robustness, which will improve cloud cover analyses and data evaluation efforts.


Total:	0
HTML:	0
PDF:	0
XML:	0