the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Brief Communication – ALARM: an innovative protocol of educating on seismic risk perception, and its assessment
Abstract. The effectiveness of risk mitigation also depends on how well-prepared and informed society is about the risk itself. The younger generation plays a key role in the scientific awareness of society, representing both the future of society and a conduit to reach and educate their families. We developed a didactic experience, based on Serious Games, dedicated to seismic risk that was tested and, then, implemented during the whole COVID-19 pandemic.
Before the start and at the end of any of these activities, an evaluation phase was carried out to assess the learning experience and the effectiveness of the science communication technique.
- Preprint
(496 KB) - Metadata XML
- BibTeX
- EndNote
Status: closed
-
RC1: 'Comment on egusphere-2022-1231', Anonymous Referee #1, 05 Dec 2022
First of all, thank you for submitting such a manuscript from the field of education. Risk perception is a very important issue and the manuscript does present a nice idea on how to foster it using GBL.
Nevertheless, there are a lot of blanks in the manuscript, which I think is partly due to the format chosen ("Brief Communication"). On the other hand, in my opinion, the work was also scientifically inadequate.
Here are my main points of concern:
- The abstract should be restructured to meet common standards (Background, Methods, Results, Conclusions). The results are missing completely.
-
A theoretical classification of the project is missing. The introduction is very general. It is not clear to what extent the project is a desideratum from previous research. Especially on GBL and Escape Rooms in tertiary education there is an incredible amount of literature. Questions arise such as: Why exactly is GBL resorted to, what advantages does the method offer in the chosen setting? Why, of all things, should GBL be used to learn about natural disasters? The Escape Room method is only defined in the practical part (from line 57), but entirely without sources. This paragraph is de facto a plagiarism.
- It is not made transparent at first what exactly the survey instrument looks like; this is only hinted at in the results (from line 144). The methods section (2.2) rather describes the basics of data analysis, for example, what a Likert scale is. This is out of place here. Does the quantitative part of the questionnaire really consist of only four items? If this is the case, all the statistical analyses presented are virtually "using a sledgehammer to crack a nut", especially since the first question is highly suggestive. Calculating Cronbach's alpha makes little sense if it is not clear exactly which construct is being measured.
- The qualitative part is not theory-based. What type of qualitative content analysis was used?
- The discussion does not include a single source and thus does not place the results in any context of previous research. Terms such as "science capital" are named, but without concrete references and without sources.
There are minor errors throughout the paper (e.g. "key-role", l. 13), a spell check is required. The sources are not formatted consistently.
Overall, the paper unfortunately does not meet the usual standards of a scientific publication. The authors may consider addressing the points I mentioned and resubmitting the paper, possibly in a journal aimed at educators.
Citation: https://doi.org/10.5194/egusphere-2022-1231-RC1 -
AC1: 'Reply on RC1', Maria Vittoria Gargiulo, 21 Mar 2023
Dear Referee #1,
thank you very much for your feedback.
Before answering your concerns point by point, please allow us to remark that the final aim of this study is not testing a new technique but developing and testing a specific protocol, which included an Escape Room, dedicated to seismic risk. We have chosen NHESS because we are geoscientists who are interested in science communication and in researching and evaluating tools to educate and raise awareness towards geoscience. Based on our experience in geoscience teaching and research activities we observed a huge gap between the public and scientific community, and we are working on finding possible tools to fill in such a gap using a scientific approach (and thus the evaluation protocol and not just the description of the experience).
In the following the comments point by point.
Reviewer: The abstract should be restructured to meet common standards (Background, Methods, Results, Conclusions). The results are missing completely.
Thank you for your comment. As you suggested we will take your suggestion into account while editing the abstract also considering the character number restraints (100 words) given by the article format.
Reviewer: A theoretical classification of the project is missing. The introduction is very general. It is not clear to what extent the project is a desideratum from previous research. Especially on GBL and Escape Rooms in tertiary education there is an incredible amount of literature. Questions arise such as: Why exactly is GBL resorted to, what advantages does the method offer in the chosen setting? Why, of all things, should GBL be used to learn about natural disasters? The Escape Room method is only defined in the practical part (from line 57), but entirely without sources. This paragraph is de facto a plagiarism
Thank you for your suggestion. For seek of readability we will edit the introduction adding a more theoretical introduction and literature on the “escape room” and Game-Based Learning. We apologise if we were not clear on the goal of the paper. Our aim is to develop and test a specific protocol, which includes an Escape Room, dedicated to seismic risk.
- Why exactly is GBL resorted to, and what advantages does the method offered in the chosen setting?
We wanted to experiment with how pupils could react to this kind of approach even in a lockdown setting. Our aim was to use a method that would allow the pupils to enjoy the experience even if they had already been in front of their screens for 6 hours for ordinary classes.
- Why, of all things, should GBL be used to learn about natural disasters?
GBL has been used in teaching and outreach for many different STEM (Fotaris and Mastoras, 2019; Lathwesen and Belova, 2021; Lopez-Belmonte et al., 2020; Veldkamp et al., 2020, 2021) for the active involvement of the participants in the experience. This approach has been proven successful for geophysics topics (Musacchio G, et al. 2022; Musacchio et al. 2015a; Musacchio et al. 2015c).
Moreover, “natural disaster” is an oxymoron. There are no disasters in nature. The “phenomenon” is natural, while a “disaster” is by “nature” anthropic. From the UNDRR site, we read that "it is time to recognise that there is no such thing as a natural disaster. [...] The world is being devastated by a mistaken notion of human progress [...] that is driving up the frequency and intensity of disaster events. [...]A hazard does not have to turn into a disaster. The decisions that we take as humans are what turn a hazard into a disaster, which is defined as a sudden calamitous event that disrupts a functioning community and exceeds their ability to cope using their own resources.” (Chmutina, K.; von Meding, J. et al. Language matters: Dangers of the “natural disaster” misnomer. Contributing Paper to the Global Assessment Report on Disaster Risk Reduction (GAR 2019), and references therein. https://www.latimes.com/archives/la-xpm-1994-02-16-me-23402-story.html https://www.jstor.org/stable/26538739)
- Finally, the paragraph you indicated is the description of our protocol, and for this reason, it doesn’t have sources.
The only line that might not seem original is the definition of “escape room”. We will rephrase the paragraph (where possible and by considering that some points are definitions).
- It is not made transparent at first what exactly the survey instrument looks like; this is only hinted at in the results (from line 144). The methods section (2.2) rather describes the basics of data analysis, for example, what a Likert scale is. This is out of place here. Does the quantitative part of the questionnaire really consist of only four items? If this is the case, all the statistical analyses presented are virtually "using a sledgehammer to crack a nut", especially since the first question is highly suggestive. Calculating Cronbach's alpha makes little sense if it is not clear exactly which construct is being measured.
Even if for only 4 questions, we do not think that using statistical analysis is “too much power” for the case. To evaluate the impact the experience had on the pupils we asked them to fill out a survey with the same questions, before and after the event. To determine in a quantitative manner whether the pre and post-answer distributions were significantly different we used a t-student test. This approach is standard and does not depend on the number of questions posed since each question is analysed separately. (Taylor, J., R. - An introduction to Error Analysis. 1997 University Science Books Ed)
Similarly, we verified the Cronbach alpha of all the data sets to ensure internal consistency of data and retrieve how closely related a set of variables is to a single indicator. In other words, we wanted to verify that the pupils’ answers were consistent. Relating only to the descriptive statistic could have given us a “false positive”.
- The qualitative part is not theory-based. What type of qualitative content analysis was used?
The paragraph related to qualitative results is only descriptive. The data have not been analysed. We apologise if this is not stated clearly and will take your comment into account when editing the final version of the manuscript.
- The discussion does not include a single source and thus does not place the results in any context of previous research. Terms such as "science capital" are named, but without concrete references and without sources.
We agree that a more extensive description of “Science capital” and “IBL” would be useful given the importance and complexity of these topics. We will edit the final version to take into account your suggestions about sources and descriptions of these topics, while still respecting the limits posed by the chosen format.
- There are minor errors throughout the paper (e.g. "key-role", l. 13), a spell check is required. The sources are not formatted consistently.
The final draft will be submitted to a certified British English proofreader to avoid typos and grammar issues.
Best regards
Citation: https://doi.org/10.5194/egusphere-2022-1231-AC1
-
RC2: 'Comment on egusphere-2022-1231', Anonymous Referee #2, 06 Feb 2023
This paper explores the use of escape rooms as an educational tool which is interesting and innovative. There are some promising results with self-reported positive experiences of the students and their interest in further studies. However it is not possible to demonstrate the educational benefit of the escape room with the current setup. In both conditions, the students attended a traditional lecture and so it is impossible to know whether any of the mentioned benefits came from the escape room. I would recommend running a further two conditions of escape room only and lecture only, in order to identify which positive impacts can be attributed to which activity.
In terms of the structure of the paper, the introduction needs to be expanded to include the background of the educational theory that has gone into this experiment. Science capital and inquiry-based learning is mentioned in the conclusion but not explained in the introduction. If these theoretical concepts / methodologies have been applied as a basis for designing their escape room this needs to be explained and linked. In summary - for what reasons do the authors think an escape room will have educational benefits (based on the existing literacture) and how are these targeted by their escape room.
Finally, there are too many grammatical errors to cover at this stage. I am conscious that the authors are based in Italy and it is unfair to expect zero grammatical errors. I would recommend re-submitting but first sending the final draft to a fluent/native speaking colleague who can proofread it beforehand.
Citation: https://doi.org/10.5194/egusphere-2022-1231-RC2 -
AC2: 'Reply on RC2', Maria Vittoria Gargiulo, 21 Mar 2023
Dear Referee #2,
thank you very much for your feedback.
Please allow us to answer your concerns:
- This paper explores the use of escape rooms as an educational tool which is interesting and innovative. There are some promising results with self-reported positive experiences of the students and their interest in further studies. However, it is not possible to demonstrate the educational benefit of the escape room with the current setup. In both conditions, the students attended a traditional lecture and so it is impossible to know whether any of the mentioned benefits came from the escape room. I would recommend running a further two conditions of escape room only and lecture only, in order to identify which positive impacts can be attributed to which activity.
Thank you for your comment, we appreciate the input and we apologise if our goal was not clear in the manuscript. Our goal is not to evaluate only if or how the escape room benefits the learning experience, we are evaluating the whole protocol: escape room + frontal lesson. We have also investigated the effects of a switch in the order of the two parts of the protocol, please see paragraph 3.1. A further, future analysis could, as the reviewer suggested, evaluate the single parts, but that is not in the scope of this brief communication.
- In terms of the structure of the paper, the introduction needs to be expanded to include the background of the educational theory that has gone into this experiment. Science capital and inquiry-based learning is mentioned in the conclusion but not explained in the introduction. If these theoretical concepts / methodologies have been applied as a basis for designing their escape room this needs to be explained and linked. In summary - for what reasons do the authors think an escape room will have educational benefits (based on the existing literacture) and how are these targeted by their escape room.
We agree that a more extensive description of “Science capital” and “IBL” would be useful given the importance and complexity of these topics. We will edit the final version to take into account your suggestions about sources and descriptions of these topics, while still respecting the limits posed by the chosen format.
As stated in the introduction, Serious Games, including also escape rooms, have been used in teaching and outreach for many different STEM (Fotaris and Mastoras, 2019; Lathwesen and Belova, 2021; Lopez-Belmonte et al., 2020; Veldkamp et al., 2020, 2021) for the active involvement of the participants in the experience. This approach has been proven successful for geophysics topics (Musacchio G, et al. 2022; Musacchio et al. 2015a; Musacchio et al. 2015c).
- Finally, there are too many grammatical errors to cover at this stage. I am conscious that the authors are based in Italy and it is unfair to expect zero grammatical errors. I would recommend re-submitting but first sending the final draft to a fluent/native speaking colleague who can proofread it beforehand.
The final draft will be submitted to a certified British English proofreader to avoid typos and grammar issues.
Best regards
Citation: https://doi.org/10.5194/egusphere-2022-1231-AC2
-
AC2: 'Reply on RC2', Maria Vittoria Gargiulo, 21 Mar 2023
Status: closed
-
RC1: 'Comment on egusphere-2022-1231', Anonymous Referee #1, 05 Dec 2022
First of all, thank you for submitting such a manuscript from the field of education. Risk perception is a very important issue and the manuscript does present a nice idea on how to foster it using GBL.
Nevertheless, there are a lot of blanks in the manuscript, which I think is partly due to the format chosen ("Brief Communication"). On the other hand, in my opinion, the work was also scientifically inadequate.
Here are my main points of concern:
- The abstract should be restructured to meet common standards (Background, Methods, Results, Conclusions). The results are missing completely.
-
A theoretical classification of the project is missing. The introduction is very general. It is not clear to what extent the project is a desideratum from previous research. Especially on GBL and Escape Rooms in tertiary education there is an incredible amount of literature. Questions arise such as: Why exactly is GBL resorted to, what advantages does the method offer in the chosen setting? Why, of all things, should GBL be used to learn about natural disasters? The Escape Room method is only defined in the practical part (from line 57), but entirely without sources. This paragraph is de facto a plagiarism.
- It is not made transparent at first what exactly the survey instrument looks like; this is only hinted at in the results (from line 144). The methods section (2.2) rather describes the basics of data analysis, for example, what a Likert scale is. This is out of place here. Does the quantitative part of the questionnaire really consist of only four items? If this is the case, all the statistical analyses presented are virtually "using a sledgehammer to crack a nut", especially since the first question is highly suggestive. Calculating Cronbach's alpha makes little sense if it is not clear exactly which construct is being measured.
- The qualitative part is not theory-based. What type of qualitative content analysis was used?
- The discussion does not include a single source and thus does not place the results in any context of previous research. Terms such as "science capital" are named, but without concrete references and without sources.
There are minor errors throughout the paper (e.g. "key-role", l. 13), a spell check is required. The sources are not formatted consistently.
Overall, the paper unfortunately does not meet the usual standards of a scientific publication. The authors may consider addressing the points I mentioned and resubmitting the paper, possibly in a journal aimed at educators.
Citation: https://doi.org/10.5194/egusphere-2022-1231-RC1 -
AC1: 'Reply on RC1', Maria Vittoria Gargiulo, 21 Mar 2023
Dear Referee #1,
thank you very much for your feedback.
Before answering your concerns point by point, please allow us to remark that the final aim of this study is not testing a new technique but developing and testing a specific protocol, which included an Escape Room, dedicated to seismic risk. We have chosen NHESS because we are geoscientists who are interested in science communication and in researching and evaluating tools to educate and raise awareness towards geoscience. Based on our experience in geoscience teaching and research activities we observed a huge gap between the public and scientific community, and we are working on finding possible tools to fill in such a gap using a scientific approach (and thus the evaluation protocol and not just the description of the experience).
In the following the comments point by point.
Reviewer: The abstract should be restructured to meet common standards (Background, Methods, Results, Conclusions). The results are missing completely.
Thank you for your comment. As you suggested we will take your suggestion into account while editing the abstract also considering the character number restraints (100 words) given by the article format.
Reviewer: A theoretical classification of the project is missing. The introduction is very general. It is not clear to what extent the project is a desideratum from previous research. Especially on GBL and Escape Rooms in tertiary education there is an incredible amount of literature. Questions arise such as: Why exactly is GBL resorted to, what advantages does the method offer in the chosen setting? Why, of all things, should GBL be used to learn about natural disasters? The Escape Room method is only defined in the practical part (from line 57), but entirely without sources. This paragraph is de facto a plagiarism
Thank you for your suggestion. For seek of readability we will edit the introduction adding a more theoretical introduction and literature on the “escape room” and Game-Based Learning. We apologise if we were not clear on the goal of the paper. Our aim is to develop and test a specific protocol, which includes an Escape Room, dedicated to seismic risk.
- Why exactly is GBL resorted to, and what advantages does the method offered in the chosen setting?
We wanted to experiment with how pupils could react to this kind of approach even in a lockdown setting. Our aim was to use a method that would allow the pupils to enjoy the experience even if they had already been in front of their screens for 6 hours for ordinary classes.
- Why, of all things, should GBL be used to learn about natural disasters?
GBL has been used in teaching and outreach for many different STEM (Fotaris and Mastoras, 2019; Lathwesen and Belova, 2021; Lopez-Belmonte et al., 2020; Veldkamp et al., 2020, 2021) for the active involvement of the participants in the experience. This approach has been proven successful for geophysics topics (Musacchio G, et al. 2022; Musacchio et al. 2015a; Musacchio et al. 2015c).
Moreover, “natural disaster” is an oxymoron. There are no disasters in nature. The “phenomenon” is natural, while a “disaster” is by “nature” anthropic. From the UNDRR site, we read that "it is time to recognise that there is no such thing as a natural disaster. [...] The world is being devastated by a mistaken notion of human progress [...] that is driving up the frequency and intensity of disaster events. [...]A hazard does not have to turn into a disaster. The decisions that we take as humans are what turn a hazard into a disaster, which is defined as a sudden calamitous event that disrupts a functioning community and exceeds their ability to cope using their own resources.” (Chmutina, K.; von Meding, J. et al. Language matters: Dangers of the “natural disaster” misnomer. Contributing Paper to the Global Assessment Report on Disaster Risk Reduction (GAR 2019), and references therein. https://www.latimes.com/archives/la-xpm-1994-02-16-me-23402-story.html https://www.jstor.org/stable/26538739)
- Finally, the paragraph you indicated is the description of our protocol, and for this reason, it doesn’t have sources.
The only line that might not seem original is the definition of “escape room”. We will rephrase the paragraph (where possible and by considering that some points are definitions).
- It is not made transparent at first what exactly the survey instrument looks like; this is only hinted at in the results (from line 144). The methods section (2.2) rather describes the basics of data analysis, for example, what a Likert scale is. This is out of place here. Does the quantitative part of the questionnaire really consist of only four items? If this is the case, all the statistical analyses presented are virtually "using a sledgehammer to crack a nut", especially since the first question is highly suggestive. Calculating Cronbach's alpha makes little sense if it is not clear exactly which construct is being measured.
Even if for only 4 questions, we do not think that using statistical analysis is “too much power” for the case. To evaluate the impact the experience had on the pupils we asked them to fill out a survey with the same questions, before and after the event. To determine in a quantitative manner whether the pre and post-answer distributions were significantly different we used a t-student test. This approach is standard and does not depend on the number of questions posed since each question is analysed separately. (Taylor, J., R. - An introduction to Error Analysis. 1997 University Science Books Ed)
Similarly, we verified the Cronbach alpha of all the data sets to ensure internal consistency of data and retrieve how closely related a set of variables is to a single indicator. In other words, we wanted to verify that the pupils’ answers were consistent. Relating only to the descriptive statistic could have given us a “false positive”.
- The qualitative part is not theory-based. What type of qualitative content analysis was used?
The paragraph related to qualitative results is only descriptive. The data have not been analysed. We apologise if this is not stated clearly and will take your comment into account when editing the final version of the manuscript.
- The discussion does not include a single source and thus does not place the results in any context of previous research. Terms such as "science capital" are named, but without concrete references and without sources.
We agree that a more extensive description of “Science capital” and “IBL” would be useful given the importance and complexity of these topics. We will edit the final version to take into account your suggestions about sources and descriptions of these topics, while still respecting the limits posed by the chosen format.
- There are minor errors throughout the paper (e.g. "key-role", l. 13), a spell check is required. The sources are not formatted consistently.
The final draft will be submitted to a certified British English proofreader to avoid typos and grammar issues.
Best regards
Citation: https://doi.org/10.5194/egusphere-2022-1231-AC1
-
RC2: 'Comment on egusphere-2022-1231', Anonymous Referee #2, 06 Feb 2023
This paper explores the use of escape rooms as an educational tool which is interesting and innovative. There are some promising results with self-reported positive experiences of the students and their interest in further studies. However it is not possible to demonstrate the educational benefit of the escape room with the current setup. In both conditions, the students attended a traditional lecture and so it is impossible to know whether any of the mentioned benefits came from the escape room. I would recommend running a further two conditions of escape room only and lecture only, in order to identify which positive impacts can be attributed to which activity.
In terms of the structure of the paper, the introduction needs to be expanded to include the background of the educational theory that has gone into this experiment. Science capital and inquiry-based learning is mentioned in the conclusion but not explained in the introduction. If these theoretical concepts / methodologies have been applied as a basis for designing their escape room this needs to be explained and linked. In summary - for what reasons do the authors think an escape room will have educational benefits (based on the existing literacture) and how are these targeted by their escape room.
Finally, there are too many grammatical errors to cover at this stage. I am conscious that the authors are based in Italy and it is unfair to expect zero grammatical errors. I would recommend re-submitting but first sending the final draft to a fluent/native speaking colleague who can proofread it beforehand.
Citation: https://doi.org/10.5194/egusphere-2022-1231-RC2 -
AC2: 'Reply on RC2', Maria Vittoria Gargiulo, 21 Mar 2023
Dear Referee #2,
thank you very much for your feedback.
Please allow us to answer your concerns:
- This paper explores the use of escape rooms as an educational tool which is interesting and innovative. There are some promising results with self-reported positive experiences of the students and their interest in further studies. However, it is not possible to demonstrate the educational benefit of the escape room with the current setup. In both conditions, the students attended a traditional lecture and so it is impossible to know whether any of the mentioned benefits came from the escape room. I would recommend running a further two conditions of escape room only and lecture only, in order to identify which positive impacts can be attributed to which activity.
Thank you for your comment, we appreciate the input and we apologise if our goal was not clear in the manuscript. Our goal is not to evaluate only if or how the escape room benefits the learning experience, we are evaluating the whole protocol: escape room + frontal lesson. We have also investigated the effects of a switch in the order of the two parts of the protocol, please see paragraph 3.1. A further, future analysis could, as the reviewer suggested, evaluate the single parts, but that is not in the scope of this brief communication.
- In terms of the structure of the paper, the introduction needs to be expanded to include the background of the educational theory that has gone into this experiment. Science capital and inquiry-based learning is mentioned in the conclusion but not explained in the introduction. If these theoretical concepts / methodologies have been applied as a basis for designing their escape room this needs to be explained and linked. In summary - for what reasons do the authors think an escape room will have educational benefits (based on the existing literacture) and how are these targeted by their escape room.
We agree that a more extensive description of “Science capital” and “IBL” would be useful given the importance and complexity of these topics. We will edit the final version to take into account your suggestions about sources and descriptions of these topics, while still respecting the limits posed by the chosen format.
As stated in the introduction, Serious Games, including also escape rooms, have been used in teaching and outreach for many different STEM (Fotaris and Mastoras, 2019; Lathwesen and Belova, 2021; Lopez-Belmonte et al., 2020; Veldkamp et al., 2020, 2021) for the active involvement of the participants in the experience. This approach has been proven successful for geophysics topics (Musacchio G, et al. 2022; Musacchio et al. 2015a; Musacchio et al. 2015c).
- Finally, there are too many grammatical errors to cover at this stage. I am conscious that the authors are based in Italy and it is unfair to expect zero grammatical errors. I would recommend re-submitting but first sending the final draft to a fluent/native speaking colleague who can proofread it beforehand.
The final draft will be submitted to a certified British English proofreader to avoid typos and grammar issues.
Best regards
Citation: https://doi.org/10.5194/egusphere-2022-1231-AC2
-
AC2: 'Reply on RC2', Maria Vittoria Gargiulo, 21 Mar 2023
Viewed
HTML | XML | Total | BibTeX | EndNote | |
---|---|---|---|---|---|
268 | 136 | 27 | 431 | 15 | 16 |
- HTML: 268
- PDF: 136
- XML: 27
- Total: 431
- BibTeX: 15
- EndNote: 16
Viewed (geographical distribution)
Country | # | Views | % |
---|
Total: | 0 |
HTML: | 0 |
PDF: | 0 |
XML: | 0 |
- 1