the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Automatic classification and segmentation of Snow Micro Penetrometer profiles with machine learning algorithms
- 1WSL Institute for Snow and Avalanche Research SLF, Flüelastrasse 11, 7260 Davos Dorf, Switzerland
- 2Institute of Cognitive Science, University Osnabrück, Wachsbleiche 27, 49090 Osnabrück, Germany
- 3Mila - Quebec AI Institute, 6666 Rue Saint-Urbain, QC H2S 3H1, Montréal, Canada
- 4School of Computer Science, McGill University, 3480 Rue University, QC H3A 2A7, Montréal, Canada
- 5Numenta, 889 Winslow Street, CA 94063, Redwood City, United States
- 1WSL Institute for Snow and Avalanche Research SLF, Flüelastrasse 11, 7260 Davos Dorf, Switzerland
- 2Institute of Cognitive Science, University Osnabrück, Wachsbleiche 27, 49090 Osnabrück, Germany
- 3Mila - Quebec AI Institute, 6666 Rue Saint-Urbain, QC H2S 3H1, Montréal, Canada
- 4School of Computer Science, McGill University, 3480 Rue University, QC H3A 2A7, Montréal, Canada
- 5Numenta, 889 Winslow Street, CA 94063, Redwood City, United States
Abstract. Snow-layer segmentation and classification is an essential diagnostic task for a wide variety of cryospheric applications. The SnowMicroPen (SMP) measures the snowpack's penetration force at submillimetre resolution against the snow depth. The resulting depth-force profile can be parameterized for density and specific surface area. However, no information on traditional snow types is currently extracted automatically. The labeling of snow types is a time-intensive task that requires practice and becomes infeasible for large datasets. Previous work showed that automated segmentation and classification is in theory possible, but can either not be applied to data straight from the field or needs additional time-costly information, such as from classified snow pits. To address this gap, we evaluate how well machine learning models can automatically segment and classify SMP profiles. We trained fourteen different models, among them semi-supervised models and artificial neural networks (ANNs), on the MOSAiC SMP dataset, a large collection of snow profiles on Arctic sea ice. We found that SMP profiles can be successfully segmented and classified into snow classes, based solely on the SMP's signal. The model comparison provided in this study enables practitioners to choose a model that is suitable for their task and dataset. The findings presented will facilitate and accelerate snow type identification through SMP profiles. Overall, snowdragon creates a link between traditional snow classification and high-resolution force-depth profiles. With such a tool, traditional snow profile observations can be compared to SMP profiles.
Julia Kaltenborn et al.
Status: final response (author comments only)
-
RC1: 'Referee Comment on egusphere-2022-938', Anonymous Referee #1, 24 Dec 2022
The manuscript presents the compilation of several machine learning algorithms for the classification of snow types of snow on Arctic sea ice from SMP penetration force profiles. The authors describe the functionality, pros and cons of each model set-up, and determine that ANNs outperform other types of supervised and semi-supervised learners at this task. An exploration of this manuscript was to determine if semi-supervised learners could accurately classify snow type from a subset of the 164 snow-type labeled SMP profiles. I found the computer science aspect of this work to be quite complete and presented at a high enough level for the comprehension of a general reader after some additional clarification on the language. As it is presented, the hypothesis is developed and tested, and a best model is determined. Discussion regarding the practitioner’s choice of model set-up within the snowdragon software package is provided for adapting this work to other SMP datasets and archived for reproducibility. I am recommending this manuscript for minor revision prior to acceptance as some aspects of the experimental design need further explanation or clarification. Addressing the provided comments may help increase the scientific quality of the manuscript, as much of the evaluation of the results remains qualitative. I do find that additional snow-related analyses and uncertainty analyses of these data and model predictions could be incorporated in this manuscript to increase the scientific significance. For this reason, I have scored Scientific Significance as “Good”. However, as the scope of the manuscript is defined, it is complete in the analysis of machine learning classification of snow type.
General and Specific comments to the authors are provided in the attached .pdf document.
-
RC2: 'Comment on egusphere-2022-938', Pascal Hagenmuller, 26 Jan 2023
Review of Automatic classification and segmentation of Snow Micro Penetrometer profiles with machine learning algorithms by Julia Kaltenborn et al.
Summary :
The authors use machine learning approaches to learn the relation between the penetration profile of the snowpack and grain shape class. The final goal is to have a quick and easy way (measurement of penetration profile with SMP) to capture one key descriptor of snow stratigraphy (grain type) which is usually time-consuming to measure. The predictors comprise different quantities derived from SnowMicroPen (SMP) measurements, including mean force, standard deviation, position in the snowpack (height), and probably noise features obtained with a statistical model (shot noise model). The goal variable is the grain shape class estimated by an expert solely from the SMP profile signature. The data comprises 164 SMP profiles measured on Arctic sea ice during winter 2019-2020 and manually labeled by a single expert. This classification leads to a segmentation of the snowpack profile into distinct layers by assuming that contiguous points of the same grain shape class belong to the same layer. The authors show that the different algorithms are able to reproduce the choice of the expert (max ROC AUC of 0.94).
Main comments:
- I am not fully getting the final objective of the paper. What is the scientific question we want to address by automatically reproducing the grain shape class inferred from the penetration profile based on undescribed expert analysis?
- By definition, the grain shape class or snow types (Fierz et al., 2009) is related to the shape of the grains and is traditionally derived from the observation of single grains on a crystal card with a magnification lens. This measurement remains manual, is very time-consuming, inevitably contains some subjectivity, and the use of classes is limited to capture the continuous nature of snow types. Trying to overcome some of the two first limitations by automatic classification is of great interest. Different attempts exist to relate the SMP signal to scalar microstructural features of snow based on the physical interpretation of the penetration process (e.g., Löwe & van Herwijnen (2012), Lin et al. (2022)) or with direct statistical / machine learning approaches (e.g., Proksch et al. (2015)). In particular, King et al. (2020) and Satyawali et al. (2009) used the latter approach to relate MEASURED grain shape class to SMP profiles. Here the ground truth is not the measured grain shape on independent data but corresponds to the interpretation of solely the SMP signal signature.
- This direct identification has never been documented so far. The description in the text is elusive, with a reference (l.76, Schneebeli et al. 1999) that does not describe the procedure. Besides, the data presented here relies on the interpretation of a single expert (l. 75-77). One cannot evaluate any reproducibility of the procedure or agreement with ground truth based on manual observation in snow pit data. Moreover, it is highly likely that the estimation is subjective. For instance, in Fig. 1, one may wonder why only the specific layer at a depth between 98 and 102 mm is labeled as « Depth hoar wind packed » and not other layers below that show similar features. In addition, there are obviously « inconsistencies in their ground truth labeling » (l. 324) and the results are linked to « different classification styles of experts » (l.332) and the evaluation is qualitative (« classification patterns […] were satisfying to domain experts » l.368). The discussion is not convincing based only on the feeling that «in the view of the authors, a temporally consistent classification is more relevant to the interpretation of the development of the snowpack, even if there is a certain, but unknown, bias to an expert interpretation » (l. 255-257). To me, it appears, in the end, that the presented algorithms are able to reproduce one analysis of one single expert on specific snowpack types. In my opinion, this limits a lot of the interest and generalization to the snow community.
- The authors refer to Fierz et al. (2009) for describing the different snow types referenced in the paper (l. 74). However, it is not very clear how the different classes presented in the paper (see legend of Fig. 1) are defined as they are not present in the international classification described in Fierz et al. (2009).
- Grain shape class has been used since the beginning of snow science and was first motivated by avalanche forecasting. It remains the most common descriptor in snowpit observations but has many known limitations. It is a discrete class whose evolution cannot be described by differential equations in models. It cannot be quickly and objectively described. Currently, the international classification is not necessarily adapted to any snow on Earth (e.g., here, the authors added classes that are not in the classification). Therefore, one may wonder why, in general, we want to stick to this description of snow.
- The interest of the algorithm is described in grandiose terms: they make « training of interdisciplinary scientists in snow type categorization obsolete » (l. 31), «can be directly employed by practitioners for their own SMP datasets in the field » (l. 250), « These findings will enable SMP practitioners to automatically analyze their SMP measurements. To that end, an SMP user must simply decide on one of the fourteen models provided » (l. 369-370). However, I do not understand these sentences. I understood that everything relies on a single expert analysis, that the model must be retrained on other data (e.g., snow data in other places around the world) and that without this expert, no model can be retrained. In contrast, the limits of previous studies are somehow presented unfairly. For instance, it is indicated that « This [generalization] would not have been possible in previous works such as Satyawali et al. (2009) since knowledge rules for one snow region and season do not transfer to other regions or seasons » (l. 335), but the exact same applies to their work as the model must be retrained in any case to be used on other snowpack climate or expert analysis in the end (the model of Satyawali et al. (2009) could be retrained too).
- The authors positively present the work as both « automatic classification and segmentation » (title and in the text) of snow profiles. It appears that no segmentation procedure is present in the paper. Indeed, the segmentation consists of saying that connected (i.e., neighboring) points with the same label belong to the same segment.
2. On the form, the description of the work is sometimes vague and incomplete.
- The objective of the paper described in l23-31 seems rather unclear to me. It took me several reads to understand that the goal is to reproduce the classification of one expert on SMP data.
- There is a welcome short bibliography on previous attempts to classify SMP profiles automatically. The description of the selected articles (Satyawali et al.,(2009), Havens et al., (2012), King et al., (2020)) would benefit from more detailed statements to capture what was really done in these papers. For instance, what is « too small to be representative » (of what?) (l. 34), « including knowledge-based rules » (l. 35), « good accuracy » (l. 42), and « additional snowpit information » (l. 42)?
- Fig. 1, the international classification (Fierz et al.,2009) provides a color code. Is there a specific reason for not using it?
- One key piece of information about the procedure is the list of predictors used as input for the ML model. They are very shortly described l. 79-86. But the description is too elusive to understand which variables are used. What are « added additional features», « time-dependent information » (where is time here ???), and « including variables of the shot noise model » (which variables?)?
Overall, I did not understand the overarching objective of the work and its applicability to different data. In addition, the presentation of previous and present work is too vague to capture the key ingredients of the methodology. I, however, acknowledge an important work to test numerous machine learning models (14) and the effort to provide the code source directly on a git repository.
I do not feel that the raised issues can be solved before publication.
Pascal Hagenmuller
Julia Kaltenborn et al.
Data sets
Snowpit SnowMicroPen (SMP) force profiles collected during the MOSAiC expedition Macfarlane, Amy R.; Schneebeli, Martin; Dadic, Ruzica; Wagner, David N.; Arndt, Stefanie; Clemens-Sewall, David; Hämmerle, Stefan; Hannula, Henna-Reetta; Jaggi, Matthias; Kolabutin, Nikolai; Krampe, Daniela; Lehning, Michael; Matero, Ilkka; Nicolaus, Marcel; Oggier, Marc; Pirazzini, Roberta; Polashenski, Chris; Raphael, Ian; Regnery, Julia; Shimanchuck, Egor; Smith, Madison M.; Tavri, Aikaterini https://doi.pangaea.de/10.1594/PANGAEA.935554
Model code and software
snowdragon Kaltenborn, Julia https://doi.org/10.5281/zenodo.7335813
Pre-trained Models for SMP Classification and Segmentation Kaltenborn, Julia; Macfarlane, Amy R.; Clay, Viviane; Schneebeli, Martin https://zenodo.org/record/7063521
Julia Kaltenborn et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 190 | 57 | 11 | 258 | 3 | 3 |
- HTML: 190
- PDF: 57
- XML: 11
- Total: 258
- BibTeX: 3
- EndNote: 3
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1