the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 07 Jul 2025
Semi-supervised machine-learning method for analyzing images from the Balloon-borne Ice Cloud particle Imager B-ICI
Abstract. Machine learning (ML) has emerged as a promising approach for image analysis, particularly for applications involving specialized or niche datasets. Different imaging techniques result in varying image characteristics and quality. ML techniques that have been previously employed to analyze ice crystals had to be developed for each instrument and imaging technique.
The Balloon-borne Ice Cloud particle Imager (B-ICI) collects and images ice particles on a film strip as it ascends through the atmosphere, producing images of ice particles on a background revealing features of the film and its oil coating. To process these raw data obtained from B-ICI, two distinct ML models were developed and trained. The first is a segmentation model designed to identify ice crystals while filtering out other objects on the image. The second is a shape classification model, which assigns the detected particles to one of four predefined categories. An intermediate step between these two models enables users to visually inspect and, if necessary, correct the segmentation output. This approach facilitates the fine-tuning of the segmentation, allowing for adjustments to model parameters as needed.
In this paper, we present these trained models and explain the validation efforts undertaken to demonstrate their applicability for analyzing data obtained from B-ICI. Comparing the model’s performance with manually analyzed new data quantifies the performance of the model. The results of this comparison shows good agreement, in particular for ⪆ 50µm.
- Preprint
(32349 KB) - Metadata XML
- BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on egusphere-2025-2818', Anonymous Referee #1, 05 Aug 2025
This work describes a machine learning (ML) algorithm that has been developed to process images captured by a balloon borne ice crystal imager (B-ICI) and classify ice crystal habit.
Overall, this manuscript does not look like it was ready for submission. While the work itself may very well be worthy of publication, the authors need to put considerably more thought into the manuscript's presentation of their work and what they want the research community to know about it. What follows below are comments to try to help the authors understand what content and clarification I feel is needed for the manuscript to be appropriate for publication, but these comments are not exhaustive. The authors should generally give more consideration to how a reader in the community would interpret their writing.
The concepts in the manuscript, particularly those that set this work apart from existing work, are not well flushed out. The manuscript needs to clearly define the context under which the work contributes something novel. This is an area where the manuscript is significantly lacking. It should then perform some sort of investigation or analysis to establish the performance of the novel aspects of the technique. This could be a validation effort (showing the algorithm does what the authors claim it does), baselining performance against a standard method or even establishing clarity about where pitfalls exist and further work is needed.
It is already noted that there have been many segmentation models developed and applied to ice crystal images, so this is not, in itself, a novel contribution. Based on the content in the manuscript, I think there are potentially two novel aspects of the work that could be flushed out. In either case the authors need to do more work defining those research elements and showing progress toward them, including addressing potential criticisms.
The authors develop a “self-supervised” machine learning approach which might be novel. This is used to augment the training dataset in an effort to make the machine learning solution more generalizable to data from other projects and even potentially instruments. I think there are some concerns with this approach, which the authors would need to address through testing and evaluation in the manuscript. My principal concern is with how this influences the sample set used for training the processor. In particular, I worry it will emphasize those particles where the CNN is already performing well and it will ignore those where it does not perform well. Wouldn’t that just reinforce its existing weaknesses? In addition, my understanding from the manuscript is that this technique did not appear to work for extending the processing algorithm to new instruments (line 225).
Another possibly novel aspect would be in regard to the authors noting the lack of machine learning algorithms that can generalize across instruments. Typically the ML architecture needs to be retrained (or even re-hyperparameter optimized) between different instruments. This work could be used to motivate the development of more generalized solutions by highlighting this challenge and showing how performance degrades in time for the same instrument or across different instruments. The work does not necessarily have to solve this problem, but rather demonstrate to the community that the problem exists, is difficult and is worth solving.
A significant issue is in the presentation of the CNN model itself where the focus tends to be poorly directed. In some cases, the manuscript focuses on unnecessary, pedantic and not entirely accurate details (like explaining convolutional layers – line 99) while poorly communicating the overall processing work flow and CNN architecture. The manuscript reads like there might just be a single scalar output (like using a series of CNN layers which feed into dense layers to output a scalar – based on the description at line 112) but the data (and use of the term “segmentation model” and pre-trained models) seems to suggest an all-convolutional NN such as a UNET architecture where the output is a 2D mask with the same dimensions as the input. I really can’t tell which it is. In another place, the role of beta in the Tversky loss function is discussed. This represents a hyperparameter in the model training process and typically would be optimized. How is this done? What value do the authors use? It’s mentioned that the model is trained with 3 values for beta but how are the results of those three outputs used and evaluated? Is there some conclusion that follows from this?There needs to be a clearer description of the data that is used for inputs and labels. What are these exactly? I’m particularly confused by the label data. Are these actual segmentation masks with dimensions the same as the input images? Is the input to the classifier the original image or the output from the particle detection step?
The paragraph in the introduction (line 37) seems to imply that the authors are addressing the problem of a generalized ML solution for processing ice particle images across instruments. My guess, as a reader, is that the “semi-supervised” approach is the method by which the authors intend to address this. But ultimately, the authors note that the method developed here does not address this problem (line 225). If I’m not interpreting this content correctly, it suggests that the authors have not been explicit enough in their description of the scope and motivation for their effort.
“Semi-supervised” is the first word in the title but there is no mention of it in the abstract or conclusions. If this represents an important feature of this work, results connected to the technique (e.g. validation or performance comparison) need to be more thoroughly explained and demonstrated. If this is not a key element of the work presented in the paper, then it should be de-emphasized and not be called out in the title. The amount of space dedicated to describing and validating the performance of the “semi-supervised” approach is also very limited. How do you know that this approach is actually improving the model? As stated earlier, I would be concerned that this method actually reinforces the limitations of the model. Can you perform an analysis that shows this method allows the gradual improvement of the model?
Citation: https://doi.org/10.5194/egusphere-2025-2818-RC1 -
RC2: 'Comment on egusphere-2025-2818', Anonymous Referee #2, 19 Aug 2025
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2025/egusphere-2025-2818/egusphere-2025-2818-RC2-supplement.pdf
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 241 | 36 | 12 | 289 | 6 | 12 |
- HTML: 241
- PDF: 36
- XML: 12
- Total: 289
- BibTeX: 6
- EndNote: 12
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1