the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 11 May 2023
Avalanche size estimation and avalanche outline determination by experts: reliability and implications for practice
Abstract. Consistent estimates of avalanche size are crucial for communicating among avalanche practitioners, but also between avalanche forecasters and the public, as for instance in public avalanche forecasts. Moreover, applications such as risk management and numerical avalanche simulations rely on accurately mapped outlines of past avalanche events. Since there is no widely applicable and objective way to measure avalanche size nor to determine the outlines of an avalanche, humans estimate these. Therefore, knowing about the reliability of avalanche size estimates and avalanche outlines is essential as errors will impact applications relying on this kind of data. Conducting three user studies, we investigate the reliability in avalanche size estimates and avalanche outlines either mapped from oblique photographs or from remotely-sensed imagery. We compare size estimates for 10 avalanches made by 170 avalanche professionals working in Europe or North America, the mappings of six avalanches from oblique photographs from 10 participants, and the mappings of avalanches visible on 2.9 km2 of remotely-sensed imagery in four different spatial resolutions from five participants. We observed an average agreement in the avalanche size estimated by the majority of respondents of 66 %, while agreement with the avalanche size considered «correct» was 74 %. Moreover, European avalanche practitioners rated avalanches significantly larger for eight out of 10 avalanches, compared to North Americans. For the outlines mapped from oblique photographs, we noted a mean overlapping proportion of 52 % for any two avalanche mappings and 60 % compared to a reference mapping. The outlines mapped from remotely-sensed imagery had a mean overlapping proportion of 46 % (image resolution 2 m) to 68 % (25 cm) between any two mappings, and 64 % (2 m) to 80 % (25 cm) when compared to the reference. Assuming that participants are equally competent in the estimation of avalanche size or the determination of avalanche outlines, we calculated a score describing the factor required to obtain the proportion of agreements between any two size estimates or overlap in avalanche areas. This factor was 0.72 for avalanche size estimates in our data set. It can be regarded as the certainty related to a size estimate by an individual, and thus provides an indication of the reliability of a label. The presented findings demonstrate that the reliability of size estimates and of mapped avalanche outlines is limited. As these data are often used as reference data or even ground truth to validate further applications, the discovered limitations and uncertainties may influence results and should be to be taken into account.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(47241 KB)
-
Supplement
(663 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(47241 KB) - Metadata XML
-
Supplement
(663 KB) - BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2023-586', Ivan Moner, 29 May 2023
The article deals with a subject of great interest in my opinion, both for the public and for snow and avalanche practitioners. It does so in a very structured way, providing new data that are treated with appropriate and rigorous statistical methods.
The methods used are explained at length, and these lead to clear results, from which important and well-justified conclusions are derived. The methods and techniques used would allow the study to be reproduced if this were of interest.
Both the title and the abstract perfectly reflect the content of the article and are easy to understand. Formulae, symbols and abbreviations are correctly explained. The same applies to the references, which are well documented.
The article is well-structured and clear. As it deals with three different experiments, it is quite long, but this is justified. Most of the figures are easy to interpret, except for Figures 2 and 3, which are practically illegible due to the small size of the fonts and photos. We suggest increasing the size of the font and photos, at the cost of eliminating the "medium size" graphics, which are also not thoroughly worked into the text.
Overall, the article is written in clear, well-structured and simple English.
In our opinion the article can be published as is, but if you wish you can take into account the following comments:
- Line 114: We find it more appropriate to speak of "public forecasting" than "regional forecasting".
- Line 121: We are a bit surprised to find 10 responses in Spain and not having known about this study before. Knowing which organizations in each country were surveyed would give information on the quality of the survey.
- Line 124: In Study 2 the sample of 9 people is obviously small. The uniform background of the respondents further weakens the results. The same happens in Study 3. The need to improve this with a larger sample could be pointed out in the conclusions.
- Line 138: Delete the first comma
- Line 190: Figure 1a, which does not exist, is cited.
- Figure 5: We are very surprised by the result shown in this figure, in which the destructive potential is the least valued of the factors. The possible causes of this anomaly are briefly discussed in the text.
- Figure A1 does not seem relevant to usIn any case, congratulations on the work done, it is excellent to us!
Citation: https://doi.org/10.5194/egusphere-2023-586-RC1 -
AC1: 'Reply on RC1', Elisabeth D. Hafner, 27 Jun 2023
Dear Ivan Moner,
thank you very much for your review and the feedback to our manuscript!
As asked for, we will improve Figure 2 and 3 in the revised version to make them well readable. Concerning your detailed comments:
- Line 114: Thank you for your feedback.
- Line 121: The survey was sent to all contacts on the EAWS mailing list as stated in the manuscript. Contacts that are not current on that list have not received the email with the link to the survey. Since we did not record any personal information, we do not know which organization our participants were a part of.
- Line 124: We have already stated the small sample as a limitation in the corresponding section (5.4) of our manuscript, we will ensure that gets clearer in the revised version.
- Line 138. We will delete the first comma in the revised version of our manuscript.
- Line 190: Thank you for pointing out this mistake. We intended to cite Figure 1. We will correct this in the revised version of our manuscript.
- Figure 5: Thank you for your comment. We were also surprised and therfore discussed possible causes.
- Figure A1: Thank you for your feedback.
Citation: https://doi.org/10.5194/egusphere-2023-586-AC1
-
AC1: 'Reply on RC1', Elisabeth D. Hafner, 27 Jun 2023
-
RC2: 'Comment on egusphere-2023-586', Brian Lazar, 07 Jun 2023
This is well excuted study yeielding important insight into an important topic. The article is well structured, clearly presented, and easy to follow. I have little in the way of constructive feedback on the explanation of methods and results. The article can be published as is, with some very minor text clean up such as Figure label corrections and extraneous commas. I've attached a commented pdf.
I do suggest this article's main strength lies in Study 1 which captures a robust set of responses from professionals across a wide range of countries. This sample size allows for more significant statistical insight, and by itself makes for a strong paper. The discusison of study 1 results describes some surprise that the destructive potential is not more heavily conisdered given that it is a principle criteria in both size scales used in the study. This is only briefly touched on in the discussion and I suggest deserves a bit more attention. I was a participant in this study, and as suggested, I believe the study design at least in part drove this outcome. The nature of the images preculudes the participant from anything better than rough guesses as destructive potential. Several of the images do not have a reference for guaging destructive potential, and 8 of the 10 images are void of trees or vegetation which help to put scales in context. This is not a criticism of the study, but would explain why destructive potential was not more frequently chose, as I recall responding much the same way. This could be an area for future investigation using a wider array of imagery.
The article is quite long with the inclusion of Study 2 and 3. While interesting, the very limited number of participants (including the authors) all with nearly identical training and background, and presumably from the same or similar organizations. This limits the import of the results. I might suggest building on the results from Studies 2 and 3, expanding the outreach, and deferring publication of these findings for a future article. This would make the submission a clear focus on Study 1 and make the article less cumbersome.
Lastly, I was curious reading the article that estimate the outline of the avalanche is covered nicely, but there is no mention of estimating the depth, particualrly of dry snow avalanches. As a practitioner, this is a critical for volume estimates. I would love to see some discussion as to how this played into size estimations.
Very well done study I think many will find enlightenting. Congratulations!
-
AC2: 'Reply on RC2', Elisabeth D. Hafner, 27 Jun 2023
Dear Brian Lazar,
thank you very much for your positive review!
We agree that the sample size of study 1 allows for the most robust analysis. We found it harder to recruit participants in study 2 and 3 because the tasks were a lot more time-consuming, and we wanted to recruit people familiar with the procedures to get representative results. Despite the described limitations we find it essential to publish these results, because in our opinion the reliability of avalanche outlines mapped has been rarely questioned and especially not investigated. We hope that future studies examining mapped avalanches will complement our study and help paint a more complete picture.
Concerning study 1, we will give more room to the discussion of the role of destructive potential in the revised version of our manuscript. We did not explicitly ask for depth estimates in this study, but we agree that it would be interesting to compare estimates on that variable.
Unfortunately, the comments in the pdf you stated was attached were not uploaded and we can currently not comment on them. We will however carefully consider them once we receive them.
Citation: https://doi.org/10.5194/egusphere-2023-586-AC2
-
AC2: 'Reply on RC2', Elisabeth D. Hafner, 27 Jun 2023
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2023-586', Ivan Moner, 29 May 2023
The article deals with a subject of great interest in my opinion, both for the public and for snow and avalanche practitioners. It does so in a very structured way, providing new data that are treated with appropriate and rigorous statistical methods.
The methods used are explained at length, and these lead to clear results, from which important and well-justified conclusions are derived. The methods and techniques used would allow the study to be reproduced if this were of interest.
Both the title and the abstract perfectly reflect the content of the article and are easy to understand. Formulae, symbols and abbreviations are correctly explained. The same applies to the references, which are well documented.
The article is well-structured and clear. As it deals with three different experiments, it is quite long, but this is justified. Most of the figures are easy to interpret, except for Figures 2 and 3, which are practically illegible due to the small size of the fonts and photos. We suggest increasing the size of the font and photos, at the cost of eliminating the "medium size" graphics, which are also not thoroughly worked into the text.
Overall, the article is written in clear, well-structured and simple English.
In our opinion the article can be published as is, but if you wish you can take into account the following comments:
- Line 114: We find it more appropriate to speak of "public forecasting" than "regional forecasting".
- Line 121: We are a bit surprised to find 10 responses in Spain and not having known about this study before. Knowing which organizations in each country were surveyed would give information on the quality of the survey.
- Line 124: In Study 2 the sample of 9 people is obviously small. The uniform background of the respondents further weakens the results. The same happens in Study 3. The need to improve this with a larger sample could be pointed out in the conclusions.
- Line 138: Delete the first comma
- Line 190: Figure 1a, which does not exist, is cited.
- Figure 5: We are very surprised by the result shown in this figure, in which the destructive potential is the least valued of the factors. The possible causes of this anomaly are briefly discussed in the text.
- Figure A1 does not seem relevant to usIn any case, congratulations on the work done, it is excellent to us!
Citation: https://doi.org/10.5194/egusphere-2023-586-RC1 -
AC1: 'Reply on RC1', Elisabeth D. Hafner, 27 Jun 2023
Dear Ivan Moner,
thank you very much for your review and the feedback to our manuscript!
As asked for, we will improve Figure 2 and 3 in the revised version to make them well readable. Concerning your detailed comments:
- Line 114: Thank you for your feedback.
- Line 121: The survey was sent to all contacts on the EAWS mailing list as stated in the manuscript. Contacts that are not current on that list have not received the email with the link to the survey. Since we did not record any personal information, we do not know which organization our participants were a part of.
- Line 124: We have already stated the small sample as a limitation in the corresponding section (5.4) of our manuscript, we will ensure that gets clearer in the revised version.
- Line 138. We will delete the first comma in the revised version of our manuscript.
- Line 190: Thank you for pointing out this mistake. We intended to cite Figure 1. We will correct this in the revised version of our manuscript.
- Figure 5: Thank you for your comment. We were also surprised and therfore discussed possible causes.
- Figure A1: Thank you for your feedback.
Citation: https://doi.org/10.5194/egusphere-2023-586-AC1
-
AC1: 'Reply on RC1', Elisabeth D. Hafner, 27 Jun 2023
-
RC2: 'Comment on egusphere-2023-586', Brian Lazar, 07 Jun 2023
This is well excuted study yeielding important insight into an important topic. The article is well structured, clearly presented, and easy to follow. I have little in the way of constructive feedback on the explanation of methods and results. The article can be published as is, with some very minor text clean up such as Figure label corrections and extraneous commas. I've attached a commented pdf.
I do suggest this article's main strength lies in Study 1 which captures a robust set of responses from professionals across a wide range of countries. This sample size allows for more significant statistical insight, and by itself makes for a strong paper. The discusison of study 1 results describes some surprise that the destructive potential is not more heavily conisdered given that it is a principle criteria in both size scales used in the study. This is only briefly touched on in the discussion and I suggest deserves a bit more attention. I was a participant in this study, and as suggested, I believe the study design at least in part drove this outcome. The nature of the images preculudes the participant from anything better than rough guesses as destructive potential. Several of the images do not have a reference for guaging destructive potential, and 8 of the 10 images are void of trees or vegetation which help to put scales in context. This is not a criticism of the study, but would explain why destructive potential was not more frequently chose, as I recall responding much the same way. This could be an area for future investigation using a wider array of imagery.
The article is quite long with the inclusion of Study 2 and 3. While interesting, the very limited number of participants (including the authors) all with nearly identical training and background, and presumably from the same or similar organizations. This limits the import of the results. I might suggest building on the results from Studies 2 and 3, expanding the outreach, and deferring publication of these findings for a future article. This would make the submission a clear focus on Study 1 and make the article less cumbersome.
Lastly, I was curious reading the article that estimate the outline of the avalanche is covered nicely, but there is no mention of estimating the depth, particualrly of dry snow avalanches. As a practitioner, this is a critical for volume estimates. I would love to see some discussion as to how this played into size estimations.
Very well done study I think many will find enlightenting. Congratulations!
-
AC2: 'Reply on RC2', Elisabeth D. Hafner, 27 Jun 2023
Dear Brian Lazar,
thank you very much for your positive review!
We agree that the sample size of study 1 allows for the most robust analysis. We found it harder to recruit participants in study 2 and 3 because the tasks were a lot more time-consuming, and we wanted to recruit people familiar with the procedures to get representative results. Despite the described limitations we find it essential to publish these results, because in our opinion the reliability of avalanche outlines mapped has been rarely questioned and especially not investigated. We hope that future studies examining mapped avalanches will complement our study and help paint a more complete picture.
Concerning study 1, we will give more room to the discussion of the role of destructive potential in the revised version of our manuscript. We did not explicitly ask for depth estimates in this study, but we agree that it would be interesting to compare estimates on that variable.
Unfortunately, the comments in the pdf you stated was attached were not uploaded and we can currently not comment on them. We will however carefully consider them once we receive them.
Citation: https://doi.org/10.5194/egusphere-2023-586-AC2
-
AC2: 'Reply on RC2', Elisabeth D. Hafner, 27 Jun 2023
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 233 | 67 | 16 | 316 | 27 | 12 | 8 |
- HTML: 233
- PDF: 67
- XML: 16
- Total: 316
- Supplement: 27
- BibTeX: 12
- EndNote: 8
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Elisabeth D. Hafner
Frank Techel
Rodrigo Caye Daudt
Jan Dirk Wegner
Konrad Schindler
Yves Bühler
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(47241 KB) - Metadata XML
-
Supplement
(663 KB) - BibTeX
- EndNote
- Final revised paper