the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 16 Apr 2024
How to communicate and educate more effectively on natural risk issues to improve disaster risk management through serious games
Abstract. This study focuses on exploring the potential of serious games for improving disaster risk management. The research involves methodological triangulation, analysing and comparing data from content analysis of serious games (6 digital games: 3 mobile apps and 3 online games), focus groups with experts and literature review. The results show that only online games fulfil the fundamental narrative indicated by the experts, with mobile apps focusing their gameplay more on interaction. Such interaction could enhance the playful aspect of the game and thus increase the desire to play; thus, the educational aspect of online games is much higher. Few online games work on issues of multiculturalism, diversity and gender. This paper provides a list of recommended features of disaster risk management games that we have categorised into three dimensions: a) character, b) information and message tone and c) narrative dynamics, reward systems and feedback. The results can be of great help to teachers and game designers in improving citizens' knowledge of disaster risk management.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(928 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(928 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-994', Miguel Angel Trejo Rangel, 22 Apr 2024
-
AC1: 'Reply on RC1', Mercedes Vázquez-Vílchez, 19 Sep 2024
Thank you for your constructive comments and suggestions. We believe addressing these comments will strengthen the paper and improve the message and key points we are trying to convey. Below, we respond to the specific comments, point by point and provide clarifications where necessary. We are confident that through this process we can improve the structure and effectiveness of the paper and communicate the results and conclusions more clearly.
Sincerely,
Dr Mercedes Vázquez Vílchez (on behalf of all co-authors)
Thank you for putting together this piece of paper. I found the topic very interesting and has large potential to be investigated to contribute to the DRM field. It is definitely an interesting topic to research considering that more and more serious games are emerging to enhance DRM.
I have some comments you can consider to improve the manuscript since I believe it can be publishable if adjustments are considered. I will go by section to make my suggestions clear enough.
** Thank you for this positive evaluation of our manuscript.
The abstract and introduction sections are clear and well-presented, but I would suggest that in the introduction you expand on any other participatory approaches that have been reported in the literature to better frame why the research team decided to focus on serious gaming. As it is framed so far you exclude other approaches which are also important such as participatory mapping, workshops, hackathons or any other in-person and online methods to engage different stakeholders.
** In the introduction, we have expanded on other participatory approaches and we clarify because we focus on serious gaming. We have introduced some sentences in the following introduction paragraph:
“Some authors suggested mutual learning, in order to promote the democratization of decisions, which combines a diverse learning methodology, such as adaptive management, experiential learning, or transformative learning (e.g. Lavell et al., 2012). Some important approaches in adaptive management incorporate the use of knowledge co-production, where scientists, politicians and other stakeholders work to interexchange, create and implement knowledge (van Kerkhoff and Lebel, 2006). In this sense, participatory mapping, workshop and hackathons are highlighted (e.g. Sullivan-Wiley et al., 2019; Trejo-Rangel et al., 2023; Macholl et al., 2023). These approaches introduce local knowledge of natural hazards into vulnerability evaluation, showing diverse vulnerabilities to natural hazards that are co-produced at local scales (Sullivan-Wiley et al., 2019). Experiential (Kolb, 1984) and transformative (Mezirow, 1995) learning remark the importance of action oriented to problem-solving, learning by-doing and how these processes create reflective thinking, theory generation and applications of knowledge, enabling behaviour change for adaptation to natural hazards (Sharpe, 2016; Lavell et al., 2012).
Following this approach, in which acquiring knowledge about natural hazards should enable citizens to make decisions and implement prevention measures, where there is recognition of active teaching methodologies, such as serious games, which may serve as a participatory and supportive tool for understanding the essential aspects of natural hazards (e.g. Solinska-Nowak et al., 2018, Tanwattana and Toyoda, 2018, Tsai et al., 2020; Schueller et al., 2020; Teague et al., 2021; Altan, et al., 2022; Villagra et al., 2023). Serious games allow users to visualise and explore phenomena that would otherwise be very difficult to experience as they enhance player immersion, and allow them to learn about the consequences of their actions at different points in time during a natural hazard (Solinska-Nowak et al., 2018; Heinzlef et al., 2024). In this way, serious games encourage experiential and transformative learning, as users try to reproduce a context as close to reality as possible that could allow to the players to enable behaviour change for adaptation and resilience to natural hazards (Villagra, 2023). The effectiveness of learning through serious games are also the immediate feedback and the emotional and sensorial experiences they provide, which is essential for learning to mitigate the effects of natural hazards (Solinska-Nowak et al., 2018; Heinzlef et al., 2024)”.
I liked how you explained serious gamin and their importance in the DRR in the theoretical framework, I think that is important considering that they have been used for some time already, however, you put together good references about them.
** Thank you for your kind words.
In the material and methods section, the section starts mentioning that it is qualitative research, however, when the results are presented, there are quantitative findings based on the questionnaires that were applied. I rather find this research as a mixed-method approach that complements findings from different methods.
** The approach to this study is qualitative. To avoid confusion, we will replace the word “questionnaire” with “semi-structured interviews”.
For the content analysis, it is mentioned that the authors designed an instrument of analysis and evaluation. However, the results mostly focus on analysis and it is not very clear to me how it complements the focus group approach which for me should be a “focus group questionnaire.
**Thank you for this comment. We will clarify this point. We will explain the content analysis in the text with the following paragraph and the Table B1 (Appendix) will appear in the main text as Table 1:
“A content analysis of the selected games was then carried out, which is a research tool used for the quantifying and analysis of the presence, meaning and relationship between specific words, themes or concepts, and thus inferences to be made about the messages within the different analysis unit (e.g. websites, journals, games, etc). The type of content analysis in this study is conceptual analysis. In conceptual content analysis a concept is chosen for examination and the analysis involves quantifying and counting its presence. It is able to identify the intentions or communicative tendencies in the games; in turn, it describes the attitudes or behaviours that result from those communications, revealing patterns in communication content. The dimensions analysed in this study were those proposed by Ouariachi et al. (2017a). These authors adapted the theoretical Social Discourse of Video Games Analysis Model (Pérez-Latorre, 2010) in an analysis instrument for games about climate change through the Delphi method. The instrument presents 51 criteria or variables, which are analysed in regards to the messages within the texts, audio, static and dynamic images of games. These criteria were classified into five dimensions: identification (features that help identify and locate the game), gameplay (set of properties that describe the player’s experience within a given game system), narrative (discursive construction around a complex phenomenon), contents (analysis of the information and messages transmitted) and educational aspects (referring to competencies, skills and learning). These criteria are described in further detail in Table 1. The analysis of the games was carried out by the authors, who played the games and filled out a form containing the criteria mentioned above.”
For the focus group (questionnaire), it is also not very clear how the MAXQDA software was used for analysing this method. I would expect the platform to be used in the content analysis section and not here. I know it is mentioned in the results, but I missed the implementation of a different method in the focus group section to gather participants' inputs. For instance, interviews or even an online workshop or so. I am concerned about conducting a questionnaire with that number of participants (8 video game experts and 14 natural hazard experts), which was not clear until I went through the results and figured it out.
**Thank you for this comment. We will clarify in the text that the semi-structured discussions were conducted online though open-ended questions. Two online semi-structured interviews were created using Google Forms, one addressed to natural hazards experts and earth science educators, and one addressed to video game experts. We will add the questions of the semi-structured interviews as supplementary materials (Tables S2 and S3). Once all the responses had been collected, codes were formulated based on their analysis using the program MAXQDA (2020) because this software allowed the authors to analyse the qualitative data collaboratively, create a common language in our codebook and reach consensus while benefiting from the unique perspectives of each team member.
I recognised that was very interesting to see how you selected the participants, but would be interesting to better understand how you approached them to answer the questionnaire.
**We will introduce details about how we approached to experts. In the text we will explain that the survey was carried out from March 10 to May 20, 2022. After sharing the interviews online, the experts in natural hazards and educators in earth sciences (13 experts) responded during the first week. However, it was necessary to insist with 42 video game experts to collect just 8 responses.
In the results section, it is not a very friendly reader to keep mentioning tables that are pages ahead of the point you are reading. It is time-consuming and you get lost while you keep going to the table and coming back to the point where you stop reading. I strongly suggest that you try to condense the results of tables B1 to B6 into one figure. I am imagining a general description of these sections, followed by a figure, design, scheme, or something creative to picture what was found, and if the reader wants to get more details, can go to the detailed tables. Please allow yourself to think about how you would like to see that in a figure that represents the important information you have gathered already. I find this section very disconnected, and it does not give an overall picture of what was found.
** We thank you for this comment. We will provide a figure (new Figure 1, see document attached) that condenses the information presented in Tables B1 to B6. Tables B1 to B6 will be included as supplementary material (Tables S4 to S8).
Also, I suggest to bring the discussions to the results section. Then, results can be compared to what has been reported in the literature. Then, you can discuss the type of related hazard, and lack of inclusivity due to the predominant languages (especially English) even when the public which could benefit from using these games does not use them, and other aspects.
** Based on your suggestion and that of the other reviewer, we will reorganize the text, and the paper will include separate sections for results, discussion, and conclusion.
Regarding the focus group results, I would try to merge both groups that participated and bring only key elements that contribute to answering your question “What are the educational and communicative elements or characteristics that serious games should have to improve DRM?”. You may notice that not all the answers from the questionnaire provide key information and you may need to prioritise what helps to respond to the question. If you decide to present the previous section (content analysis) on how serious games are in a more condensed way, you could take advantage of that and highlight how they “should improve” with these findings. I can see that the data collected in the content analysis could help to build a current picture and the information collected in questionnaires would be the transformed version of that.
**Thank you for this comment. We will eliminate those answers of the questionnaires that fail to provide key information for answering the research question “What are the educational and communicative elements or characteristics that serious games should have to improve DRM?”. We will delete the responses from videogame experts on curiosity and chance (line 291 to 298 in older version) and the corresponding older Figure 2. In this way, we will highlight those key answers directly related to the summary figure of content analysis of the games evaluated (new figure 1).
Some charts and tables look a bit unnecessary since they do not say so much and take up a large space in the document. Please reconsider which of them are useful and how would be a better way to present more connected results where discussion can be included. I do see that you have collected valuable information, I just do not get how the results are organised, I believe this section should be better presented.
**Thank you for this comment. We will reorganise the information presented. We will incorporate the key information of the older Figures 1, 3 and 4 in Tables 2 and 3 (categories of the expert responses) and, therefore, these older figures will be deleted.
I suggest leaving the conclusions as a separate section without merging it with the discussions since I see a lot more potential to include discussions in the results section.
**We now present the results, discussion and conclusions in separate sections in the paper.
References
Altan, B., Gürer, S., Alsamarei A., Kıvılcım Demir, D., Şebnem Düzgün, H., Erkayaoğlu, M.. Surer, E.: Developing serious games for CBRN-e training in mixed reality, virtual reality, and computer-based environments, Int. J. Disaster Risk Reduct., 77, 103022, https://doi.org/10.1016/j.ijdrr.2022.103022, 2022.
Heinzlef, C., Lamaury, Y., Serre, D.: Improving climate change resilience knowledge through a gaming approach: Application to marine submersion in the city of Punaauia, Tahiti, Environmental Advances, 15, 100467, 2024, 10.1016/j.envadv.2023.100467. hal-04353914.
Kolb, D.A.: Experiential Learning: Experience as the Source of Learning and Development. Prentice-Hall, Englewood Cliffs, NJ., 1984.
Lavell, A., Oppenheimer, M., Diop, C., Hess, J., Lempert, R., Li, J., Muir-Wood, R., Myeong, S., Moser, S., Takeuchi, K., Cardona, O. D., Hallegatte, S., Lemos, M., Little, C., Lotsch, A., and Weber, E.: Climate change: new dimensions in disaster risk, exposure, vulnerability, and resilience, in: Managing the risks of extreme events and disasters to advance climate change adaptation. A special report of working groups I and II of the Intergovernmental Panel on Climate Change (IPCC), edited by: Field, C.B., Barros, V., Stocker, T.F., Qin, D., Dokken, D.J., Ebi, K.L., Mastrandrea, M.D., Mach, K.J., Plattner, G.K., Allen, S.K., Tignor, M., Midgley P.M., Cambridge University Press, Cambridge and New York, 25–64, 2012.
Macholl, J.D., Roberts, H., Steptoe, H, Sun, S., Angus, M., Davenport, C., Luscombe, W., Rolker, H.B., Pope, E.C.D., Dawkins, L.C., Munday, G., Giles, D., Lam, T., Deutloff, J., Champion, A.J., Bloomfield, H.C., Mendes, J., Speight, L., Bradshaw, C.D., Wyatt, F. : A collaborative hackathon to investigate climate change and extreme weather impacts in justice and insurance settings, Weather, 79(6), 196-203., 2024.
Mezirow, J.: Transformation theory in adult learning. In: Defense of the Life World [Welton, M.R. (ed.)]. State University of New York Press, Albany, NY, 39-70, 1995.
Ouariachi, T.; Gutiérrez-Pérez, J.; Olvera-Lobo, M.D: Criterios de evaluación de juegos online sobre cambio climático: aplicación del método Delphi para su identificación. Rev. Mex. Investig. Educ., 22, 445–474, 2017a.
Pérez-Latorre, O.: Análisis de la significación del videojuego: Fundamentos teóricos del juego, el mundo narrativo y la enunciación interactiva como perspectivas de estudio del discurso (Analisis of video game signification: game theoretic fundamentals, the narrative world and the interactive enunciation as perspectives for the discourse study) (Unpublished doctoral dissertation). Barcelona, Spain: Universitat Pompeu Fabra, Departament de Comunicació, 2019.
Schueller, L., Booth, L., Fleming, K., Abad, J.: Using serious gaming to explore how uncertainty affects stakeholder decision-making across the science-policy divide during disasters, Int. J. Disaster Risk Reduct., 51, 101802, https://doi.org/10.1016/j.ijdrr.2020.101802, 2020.
Sharpe, J: Understanding and unlocking transformative learning as a method for enabling behaviour change for adaptation and resilience to disaster threats, Int. J. Disaster Risk Reduct, 17, 213-219, 2023.
Solinska-Nowak, A., Magnuszewski, P., Curl, M., French, A., Keating, A., Mochizuki, J., Liu, W., Mechler, R., Kulakowska, M., Jarzabek, L.: An overview of serious games for disaster risk management – prospects and limitations for informing actions to arrest increasing risk, Int. J. Disaster Risk Reduct., 31, 1013–1029, https://doi.org/10.1016/j.ijdrr.2018.09.001, 2018.
Sullivan-Wileya K.A., Short-Gianottib, A.G., Casellas Connorsc, J.P.: Mapping vulnerability: Opportunities and limitations of participatory community mapping, Applied Geography, 105: 47–57, 2019.
Tanwattana, P., Toyoda, Y.: Contributions of gaming simulation in building community-based disaster risk management applying Japanese case to flood prone communities in Thailand upstream area, Int. J. Disaster Risk Reduct., 27, 199-213, https://doi.org/10.1016/j.ijdrr.2017.10.007, 2018.
Teague, A., Sermet, Y., Demir, I., Muste, M.: A collaborative serious game for water resources planning and hazard mitigation, Int. J. Disaster Risk Reduct., 53, 101977, https://doi.org/10.1016/j.ijdrr.2020.101977, 2021.
Trejo-Rangel, M.A, Marchezini, V., Rodriguez, D.A., Messias dos Santos, D., Gabos, M, Lélis de Paula, A, Santos, E., Sampaio do Amaral, F.: Incorporating social innovations in the elaboration of disaster risk mitigation policies, Int. J. Disaster Risk Reduct, 84, 103450, 2023.
Tsai, M.H., Chang, Y.L., Shiau, J.S., Wang, S.M.: Exploring the effects of a serious game-based learning package for disaster prevention education: The case of Battle of Flooding Protection, Int. J. Disaster Risk Reduct., 43, 101393, https://doi.org/10.1016/j.ijdrr.2019.101393, 2020.
van Kerkhoff, L., L. Lebel: Linking knowledge and action for sustainable development. Annual Review of Environment and Resources, 31,445-477, 2006.
Villagra, P., Peña y Lillo, O., Ariccio, S., Bonaiuto, M. and Olivares-Rodríguez, C.: Effect of the Costa Resiliente serious game on community disaster resilience, Int. J. Disaster Risk Reduct., 91, 103686, https://doi.org/10.1016/j.ijdrr.2023.103686, 2023.
-
AC1: 'Reply on RC1', Mercedes Vázquez-Vílchez, 19 Sep 2024
-
RC2: 'Comment on egusphere-2024-994', Anonymous Referee #2, 08 Jul 2024
General comments
This paper examines the potential benefits of serious games for enhancing disaster risk management. The researchers employed a methodological triangulation approach, which involved content analysis, focus group discussions with experts and literature review. The findings show that online games are more successful in meeting experts' narrative criteria and have higher educational value, while mobile apps focus more on interaction, increasing engagement but lacking educational depth. The study also highlights a lack of attention to multiculturalism, diversity, and gender issues.
Thank you for your manuscript. The research questions posed are interesting and the research, per se, is relevant to the research area. However, the paper’s structure and the presentation of the information lack clarity and the grammar used is not at a level for publication.
- The grammar in the paper is seriously lacking throughout major parts of the paper. Therefore, please have the paper be proofread by a native speaker.
- The introduction would benefit from a clearer delineation of the research gap(s) and how this paper specifically is trying to answer it/them.
- I would appreciate it if the research questions was mentioned in the introduction and then specifically taken up again in the discussion part and answered in more detail
- Explain in more detail how the information in the results section were obtained. Adding the questionnaire to the appendix could be helpful to remedy that and help readers understand what the questionnaire looked like.
- consider if all of the figures presented in the results section are actually necessary or if the information presented in them could be condensed into one or two figure(s). Also rethink how the information in the tables is presented (see specific comment below)
- Some parts of the results section rather belongs to the discussion and conclusion chapter, i.e. leave all interpretation of the results to the discussion. Some, but by far not all, of the instances are commented on in the specific comments (see specific comment below)
- I would separate the discussion and conclusion section so as to make it more reader-friendly
- the paper could benefit from a clearer use of terminology = unify the use of i.e. natural risks, natural hazards, natural hazards-related disaster
- refrain from using evaluating language if it is not language used by the experts (and if the latter, use direct quotes)
Specific comments:
Section 2 (Theoretical framework):
Line 99: add percentage for the prevalence of storms
Section 3 (Materials and methods):
Line 135: the age range is unclear – if the game is geared at young people without giving an age upper limit, it should rather be argued that it is for young and adult people
Line 137: it would be helpful for the reader to know on the basis of which criteria this selection was made
Line 140ff: describe in more detail what the content analysis looked like with games as objects of the study. How was content extracted from the games, which aspects of the structure of the games were analyzed (i.e. visual, audio, text, etc.) and how. How did you deal with the interactive aspect of the games, i.e. different outcomes of the games depending on the players’ decisions, etc.?
Line 144: “First” is not needed if there are no other arguments listed (“second”, “third”)
Line 147: fix reference (only year in brackets)
Line 152: close bracket
Lines 163-165: Rewrite accordingly: This study involved individuals with at least 5 years of professional or experiential knowledge of the research topic, constituting an informed panel and thereby justifying the use of the title "experts" (Mullen, 2003).
Line 170: include an explanation of how these participants are experts in natural hazards, i.e. are they researchers, practitioners, stakeholders…
Line 179: add citation for MAXQDA program. Also, it is unclear what the program was used for.
Section 4 – Results:
Clarify how this information was obtained. Were the following questions posed as open questions, was there a selection of answers given and could experts add own ideas, …? As mentioned above, adding the appendix to the paper helps readers understand how this information was elicited. This applies to all other questions in the following tables in the result section.
Rethink the use of figures (see comment above) and consider condensing some figures into one. Also, the tables do not present information in a very reader-friendly way, consider rephrasing the items shown so it is more clear to the readers what they mean.
Line 188: To me, there seems to be no point in highlighting Hungarian and Turkish as available languages for the paper at hand, so delete and rather than pointing out specific languages maybe state the number of languages in which the game is available
Line 221: add why the term “natural risk” in particular is one you looked out for. Also I would, again, argue that this is content for the discussion section of the paper
Line 221-222: explain in more detail – why is this an interesting result and what constitutes a complex concept (consider also giving examples). Again, this would be content for the discussion section.
Line 231: Consider moving this whole subsection (4.1.4) to the beginning of the chapter. Painting a picture of the games’ setup could help readers in getting an understanding of the games and differentiating them better
Line 233: use “in multi-player mode” instead of “collective way”
Line 238: make the link clearer as to why the online games offer educational advantages over the mobile ones. What characterizes the mobile ones in comparison?
Line 243: Clarify what you mean by “the other games might also be played in an hour”
Line 245: Clarify what the difference is between positive feedback and a reward system
Line 250: Explain what Bloom’s Taxonomy is and how this is relevant here
Line 258f: Clarify if only online games dealt with these dimensions
Line 266ff/Table 1: clarify what “Always” means in “reward systems” section, especially since this is not referred to in the text. Also consider renaming this item as it is too unspecific
Line 266ff/Table 1: clarify why progress and motivation is handled as one category. Consider renaming the third category to “motivate” in order to make terminology uniform (i.e. all verb infinitives or the gerund). Do this for the entire table.
Line 266ff/Table 1: the last 3 subcategories in the “duration” section are confusing to me as their connection to the theme of duration only makes sense after reading the text. This, to me, raises the question if the information should rather be presented in an answer-like style, i.e. “game should not take more than an hour” “game should take up to several hours”, “game length can be variable if narrative is employed”, etc. Such rephrasing could help readers understand the subcategories better.
Line 280: 46% does not appear in the table. Check if numbers in text and table align.
Line 291: Figure 1 is not referred to in the text and, even if it were, seems unnecessary – information depicted does not need its own figure
Line 293: “All of the experts agree” does not fit to the number in the cited figure (figure only shows 62% for curiosity).
Lines 295ff: the last to sentences of the paragraph seem to repeat the same information.
Line 301: Unsure whether this conclusion can be drawn from merely 7% and 21% of responses.
Line 302f: This is an interpretation, move this part to the discussion section.
Table 2: there is a typo in “multiculturalis”
Table 2: change phrasing of “normal person” – avoid evaluative language if it is not language explicitly used by the experts. If that were the case, use direct quotes
Figure 5: figure is cut off on the left
Line 363f: cut “wonderful” and “great” - avoid evaluative language if it is not language explicitly used by the experts. If that were the case, use direct quotes
Section 5 (discussion and conclusion):
Move all interpretation of the obtained data currently found in the results section here and consider splitting discussion and conclusion into two separate chapters
Lines 402ff: rephrase this sentence as it is not clear which of the aspects named are those of “secondary presence”
Section 6 (limits and recommendation for future studies):
This section requires more detail and clarity, i.e. what aspects of the methodological approach of triangulation helps in preventing a subjective interpretation of the results? Clarify what you mean by “seeking consensus”. Give more detailed examples of aspects to be studies for future research.
Appendix:
Table B3 – Disaster Master/global history: rearrange this paragraph, moving the last sentence to the front (“the overarching framework …”)
Table B3 – Earthquake Relief Rescue+/Representation of character: cut out “but in a very impersonal manner” – avoid evaluating language
Table B6 – Competencies: include a footnote explaining the abbreviations
Table B6 – Learning curve: consider replacing “learning curve” with “level of difficulty”
Table B6 – Accessibility: Explain what is meant by accessibility
Citation: https://doi.org/10.5194/egusphere-2024-994-RC2 -
AC2: 'Reply on RC2', Mercedes Vázquez-Vílchez, 19 Sep 2024
Thank you for your constructive comments and suggestions. We believe addressing these comments will strengthen the paper and improve the message and key points we are trying to convey. Below, we respond to the specific comments, point by point and provide clarifications where necessary. We are confident that through this process we can improve the structure and effectiveness of the paper and communicate the results and conclusions more clearly.
Sincerely,
Dr Mercedes Vázquez Vílchez (on behalf of all co-authors)
This paper examines the potential benefits of serious games for enhancing disaster risk management. The researchers employed a methodological triangulation approach, which involved content analysis, focus group discussions with experts and literature review. The findings show that online games are more successful in meeting experts' narrative criteria and have higher educational value, while mobile apps focus more on interaction, increasing engagement but lacking educational depth. The study also highlights a lack of attention to multiculturalism, diversity, and gender issues.
Thank you for your manuscript. The research questions posed are interesting and the research, per se, is relevant to the research area. However, the paper’s structure and the presentation of the information lack clarity and the grammar used is not at a level for publication.
** We thank the reviewer for the careful review. We agree that the paper will greatly benefit from modifying the structure, improving the presentation of the information, and enhancing the grammar.
- The grammar in the paper is seriously lacking throughout major parts of the paper. Therefore, please have the paper be proofread by a native speaker.
** A native speaker will revise the paper.
- The introduction would benefit from a clearer delineation of the research gap(s) and how this paper specifically is trying to answer it/them.
**Thank you for this comment. We will clarify the research gaps in the introduction section. The text will include a new sentence in the following paragraph: “However, while serious games can contribute by giving researchers useful evidence into how people conceive disasters, there is a poor understanding of the representation of catastrophes within popular culture (Gampell and Gaillard, 2016; Safran et al., 2024). Some authors related the characteristics of several disaster games to the disaster risk reduction framework (mitigation, preparedness and recovery), highlighting the need for further research into how game characteristics (mechanics, dynamics, narrative and content), player skills, motivations and social interactions contribute towards improving decision-making in the area of disaster risk reduction (DRR) (Gampell and Gaillard, 2016; Safran et al., 2024). Few works address the influence of video games on the tendency of players to prepare for natural hazards (Tanes and Cho, 2013; Tanes, 2017; Safran et al, 2024). Attention is drawn to the lack of solid scientific evidence of the potential of serious games, with challenges remaining for the development of more detailed studies to test and demonstrate the effectiveness of serious games for DRM education (Weyrich et al., 2021; Safran et al., 2024).
- I would appreciate it if the research questions was mentioned in the introduction and then specifically taken up again in the discussion part and answered in more detail.
**Thank you for this comment. We will specify the research question in the introduction. The following paragraph will appear in the text: “This paper aims to explore the potential of serious games for improving DRM. The main research question it raises is: How can we educate and communicate more effectively about natural hazards through serious games? To this end, the following research sub-questions are posed: (a) How do serious games communicate and educate on issues related to natural hazards? and (b) Which educational and communicative elements or characteristics should serious games have to improve DRM? This work will follow a methodology consisting of triangulation-based qualitative research. The research method includes a content analysis of selected serious games applied to DRM, a focus group of experts and a literature review (the constructs of the investigation were determined with the help of introductory literature)”. Also we will indicate the research question in the discussion to provide a more detailed answer.
-Explain in more detail how the information in the results section were obtained. Adding the questionnaire to the appendix could be helpful to remedy that and help readers understand what the questionnaire looked like.
**The questions of the semi-structured interviews will be added as supplementary materials (Table S2 and S3).
-consider if all of the figures presented in the results section are actually necessary or if the information presented in them could be condensed into one or two figure(s). Also rethink how the information in the tables is presented (see specific comment below).
**We agree that not all figures presented are necessary. We will incorporate the key information from Figures 1, 3, and 4 into Tables 2 and 3 (expert response categories) and, therefore, these figures will be deleted. Figure 2 will also be eliminated based on feedback from another reviewer.
-Some parts of the results section rather belongs to the discussion and conclusion chapter, i.e. leave all interpretation of the results to the discussion. Some, but by far not all, of the instances are commented on in the specific comments (see specific comment below).
**Thank you for this comment. The interpretations will be moved from the results to the discussion and conclusion sections.
- I would separate the discussion and conclusion section so as to make it more reader-friendly.
**We appreciate the suggestion and will separate the discussion and conclusions.
- the paper could benefit from a clearer use of terminology = unify the use of i.e. natural risks, natural hazards, natural hazards-related disaster.
** We will unify the terminology using only natural hazards.
- refrain from using evaluating language if it is not language used by the experts (and if the latter, use direct quotes).
**We thank you for this comment. We use the language used by the experts. However, we acknowledge the potential confusion and we will add two tables with responses of the experts as supplementary material (Tables S9 and S10, see attached document).
Specific comments:
Section 2 (Theoretical framework):
Line 99: add percentage for the prevalence of storms.
**We will add 34% as a percentage for the prevalence of storms.
Section 3 (Materials and methods):
Line 135: the age range is unclear – if the game is geared at young people without giving an age upper limit, it should rather be argued that it is for young and adult people.
**We will clarify that the games evaluated are aimed at young people and adults (age 12 and upwards).
Line 137: it would be helpful for the reader to know on the basis of which criteria this selection was made.
**We will remark that according to the criteria explained previously, in this phase, from 17 digital games, a total of 6 games were selected: 3 mobile apps (Earthquake Relief Rescue, Geostorm and Disaster Rescue Service) and 3 online games (Build a kit, Disaster Master and Stop Disasters).
Line 140ff: describe in more detail what the content analysis looked like with games as objects of the study. How was content extracted from the games, which aspects of the structure of the games were analyzed (i.e. visual, audio, text, etc.) and how. How did you deal with the interactive aspect of the games, i.e. different outcomes of the games depending on the players’ decisions, etc.?
** Thank you for this comment. We will clarify this point. We will explain the content analysis in the text with the following paragraph and the Table B1 (Appendix) will appear in the main text as Table 1:
“A content analysis of the selected games was then carried out, which is a research tool used for the quantifying and analysis of the presence, meaning and relationship between specific words, themes or concepts, and thus inferences to be made about the messages within the different analysis unit (e.g. websites, journals, games, etc). The type of content analysis in this study is conceptual analysis. In conceptual content analysis a concept is chosen for examination and the analysis involves quantifying and counting its presence. It is able to identify the intentions or communicative tendencies in the games; in turn, it describes the attitudes or behaviours that result from those communications, revealing patterns in communication content. The dimensions analysed in this study were those proposed by Ouariachi et al. (2017a). These authors adapted the theoretical Social Discourse of Video Games Analysis Model (Pérez-Latorre, 2010) in an analysis instrument for games about climate change through the Delphi method. The instrument presents 51 criteria or variables, which are analysed in regards to the messages within the texts, audio, static and dynamic images of games. These criteria were classified into five dimensions: identification (features that help identify and locate the game), gameplay (set of properties that describe the player’s experience within a given game system), narrative (discursive construction around a complex phenomenon), contents (analysis of the information and messages transmitted) and educational aspects (referring to competencies, skills and learning). These criteria are described in further detail in Table 1. The analysis of the games was carried out by the authors, who played the games and filled out a form containing the criteria mentioned above.”
Line 144: “First” is not needed if there are no other arguments listed (“second”, “third”).
**We thank you for this comment. We will modify this paragraph and the first word will be deleted.
Line 147: fix reference (only year in brackets).
**We will correct this reference.
Line 152: close bracket.
** We will close the brackets.
Lines 163-165: Rewrite accordingly: This study involved individuals with at least 5 years of professional or experiential knowledge of the research topic, constituting an informed panel and thereby justifying the use of the title "experts" (Mullen, 2003).
**Thank you for this comment. We will properly rewrite this sentence.
Line 170: include an explanation of how these participants are experts in natural hazards, i.e. are they researchers, practitioners, stakeholders…
**We will add the following explanation to the paper: As regards the surveys of experts in natural hazards, they present different natural hazard backgrounds (climate change, floods, earthquakes, volcanoes and mass movement) and professions (researchers, emergencies experts, a politician and the partner of a consulting company in urban and territorial planning).
Line 179: add citation for MAXQDA program. Also, it is unclear what the program was used for.
**Thank you for this comment. We will include the correct reference; MAXQDA (2020) will appear in the text and we will explain that this qualitative data analysis software allowed the authors to analyse the data collaboratively, create a common language in our codebook and reach consensus while benefiting from the unique perspectives of each team member.
Section 4 – Results:
Clarify how this information was obtained. Were the following questions posed as open questions, was there a selection of answers given and could experts add own ideas, …? As mentioned above, adding the appendix to the paper helps readers understand how this information was elicited. This applies to all other questions in the following tables in the result section.
**Thank you for this comment. The semi-structured discussions were conducted online though open-ended questions. We will add the questions of these as supplementary materials (Table S2 and S3).
Rethink the use of figures (see comment above) and consider condensing some figures into one. Also, the tables do not present information in a very reader-friendly way, consider rephrasing the items shown so it is more clear to the readers what they mean.
**We will incorporate the key information from Figures 1, 3, and 4 into Tables 2 and 3 (expert response categories) and, therefore, these figures will be deleted. Figure 2 will also be eliminated based on feedback from another reviewer. We will provide a figure (new Figure1, see document attached) that condenses the information presented in Tables B1 to B6. Tables from B1 to B6 will be included as supplementary material (Tables S4 to S8). In order to improve the understanding of the tables some categories will be rephrased.
Line 188: To me, there seems to be no point in highlighting Hungarian and Turkish as available languages for the paper at hand, so delete and rather than pointing out specific languages maybe state the number of languages in which the game is available.
**We will specify that the Geostorm game gives the option of 11 languages.
Line 221: add why the term “natural risk” in particular is one you looked out for. Also I would, again, argue that this is content for the discussion section of the paper.
*We thank you for this comment. We will delete the following sentence: “We did not find the term "natural risk", but rather "natural catastrophe" or "natural disaster", because in the text it is enough clear that the games use rather alarmist terms such as "emergency", "catastrophe" and "disaster”.
Line 221-222: explain in more detail – why is this an interesting result and what constitutes a complex concept (consider also giving examples). Again, this would be content for the discussion section.
**We agree, and this sentence will be moved to discussion section. We will add examples of complex concepts such as earthquake epicentre, floodplain, etc.
Line 231: Consider moving this whole subsection (4.1.4) to the beginning of the chapter. Painting a picture of the games’ setup could help readers in getting an understanding of the games and differentiating them better.
**We will relocate the gameplay subsection to the beginning of the chapter. We will provide a figure (new Figure1, see attached document) that condenses the information presented from Tables B1 to B6, including Table B5 (game play dimension).
Line 233: use “in multi-player mode” instead of “collective way”.
**We agree, we will use multi-player mode.
Line 238: make the link clearer as to why the online games offer educational advantages over the mobile ones. What characterizes the mobile ones in comparison?.
**The sentence: “Therefore, these three online games offer further educational advantages over the mobile app examples”will be deleted. This idea will be integrated into the discussion section.
Line 243: Clarify what you mean by “the other games might also be played in an hour”.
**Thank you for this comment. The sentence will be modified as: “The other games present levels that might also be played in one hour”.
Line 245: Clarify what the difference is between positive feedback and a reward system.
** We will clarify this point through the following addition to the text : “Positive feedback, though messages the player receives in light of certain actions, are abundant in the games, and we only found a reward system in the online examples”. In addition, this point is extensively explained in Table B1 (Table1 in new version).
Line 250: Explain what Bloom’s Taxonomy is and how this is relevant here.
**Thank you for this comment. We will explain that Bloom's Taxonomy consists of a hierarchical structure of objectives or levels that allow educators to evaluate the learning process of students; it is also a useful starting point for designing activities to achieve meaningful and lifelong learning. Accordingly, the evaluation criteria related to "Remember" and "Understand" are classified as "Basic"; the criteria related to "Apply" and "Analyse" are catalogued as "Optimal"; and the criteria related to "Evaluate" and "Create" are classified as "Desirable". Taking into account these levels, the most complete are Stop Disasters, Disaster Master and Geostorm which cover all of them; however, Earthquake Relief Rescue and Disaster Rescue Service are the two games that cover only the Basic and Optimal learning levels."
Line 258f: Clarify if only online games dealt with these dimensions.
**We will clarify that the Multiculturalism and Cultural Awareness and Expression Competences are addressed only by the online games (Build a kit, Disaster Master and Stop Disasters).
Line 266ff/Table 1: clarify what “Always” means in “reward systems” section, especially since this is not referred to in the text. Also consider renaming this item as it is too unspecific.
**Based on the other reviewer, this category will be eliminated from Table 1 (Table 2 in revised version), because it does not contribute towards answering the research question. In addition, the idea expressed by the experts in the recommended/satisfactory category is too similar to the Always category.
Line 266ff/Table 1: clarify why progress and motivation is handled as one category. Consider renaming the third category to “motivate” in order to make terminology uniform (i.e. all verb infinitives or the gerund). Do this for the entire table.
**According to the suggestion of the other reviewer, these categories from the dimension level will be eliminated from Table 1 (Table 2 in the new version), as they do not contribute towards answering the research question.
Line 266ff/Table 1: the last 3 subcategories in the “duration” section are confusing to me as their connection to the theme of duration only makes sense after reading the text. This, to me, raises the question if the information should rather be presented in an answer-like style, i.e. “game should not take more than an hour” “game should take up to several hours”, “game length can be variable if narrative is employed”, etc. Such rephrasing could help readers understand the subcategories better.
**Based on the suggestion of the other reviewer, the categories 'narrative' and 'engage' will be eliminated from Table 1 (Table 2 in new version) as they do not contribute towards answering the research question. We will add new tables with examples of expert responses in the supplementary materials to improve the understanding of the remaining categories (Tables S9 and S10, see attached documents).
Line 280: 46% does not appear in the table. Check if numbers in text and table align.
**We thank you for this comment. We will correct the data.
Line 291: Figure 1 is not referred to in the text and, even if it were, seems unnecessary – information depicted does not need its own figure.
**Figure 1 will be deleted and the data will appear in Table 1 (Table 2 in the new version).
Line 293: “All of the experts agree” does not fit to the number in the cited figure (figure only shows 62% for curiosity).
**According to the other reviewer, we will eliminate those answers of the semi-structured interviews that do not provide key information towards answering the research question: What are the educational and communicative elements or characteristics that serious games should have to improve DRM? We will delete the responses from videogame experts on curiosity and chance (line 291 to 298) and we will delete the corresponding Figure 2. In this way, we will highlight those key answers directly related to the content analysis of the games evaluated.
Lines 295ff: the last to sentences of the paragraph seem to repeat the same information.
**As we mentioned in the previous point, we will delete the responses from videogame experts on curiosity and chance (line 291 to 298).
Line 301: Unsure whether this conclusion can be drawn from merely 7% and 21% of responses.
**We thank you for this comment. We will delete the categories fun and engage, corresponding to 7% and 21% of the level of interaction dimension. We will include the level of interaction dimension with only one category (narrative) in Table 1 (Table 2 in the new version).
Line 302f: This is an interpretation, move this part to the discussion section.
**We appreciate the suggestion and we will modify this sentence as: “However, the code most represented in this question is narrative, since in a serious game with strong narrative the interaction can be lower and they can have a great impact.” We will add two tables in supplementary materials showing the experts’ answers (see document attached).
Table 2: there is a typo in “multiculturalis”.
**We will correct this typo.
Table 2: change phrasing of “normal person” – avoid evaluative language if it is not language explicitly used by the experts. If that were the case, use direct quotes.
**We thank you for this comment. We do not use evaluative language; instead, we use the language explicitly employed by the experts. We will thoroughly review the results to relocate any sentences that include the authors' interpretations. We will add two tables with direct quotes from experts to help readers understand the results (Tables S9 and 10, see documents attached).
Figure 5: figure is cut off on the left.
**We will correct this in the revised version.
Line 363f: cut “wonderful” and “great” - avoid evaluative language if it is not language explicitly used by the experts. If that were the case, use direct quotes.
**We appreciate the suggestion and, in this case, we will delete these words from the text to avoid evaluative language.
Section 5 (discussion and conclusion):
Move all interpretation of the obtained data currently found in the results section here and consider splitting discussion and conclusion into two separate chapters.
**We will move all interpretations from the results section to a new discussion section. We will provide a separate section for the conclusion.
Lines 402ff: rephrase this sentence, as it is not clear which of the aspects named are those of “secondary presence”.
**We will rephrase this sentence. The new version will include: “These desirable characteristics are: exploratory characters in a cooperative dynamic, with simple and non-technical narratives, based on academic sources, multiculturalism, consideration of diversity and gender, fun, short games, constructive feedback system, while the rewards could have a secondary presence.”
Section 6 (limits and recommendation for future studies):
This section requires more detail and clarity, i.e. what aspects of the methodological approach of triangulation helps in preventing a subjective interpretation of the results? Clarify what you mean by “seeking consensus”. Give more detailed examples of aspects to be studies for future research.
** We will rewrite this section. In the new version of the paper it will appear as:
“The limitations of this study are related to the subjective nature of qualitative analyses, due to their focus on interpreting meaning and the meaning-making process. Consequently, a methodical triangulation approach (content analysis of selected serious games, focus group with experts and literature review) was used to minimize this limitation (Ouariachi et al., 2017b).
On the other hand, in section 3.2 Focus group with experts (3. Materials and methods) we will clarify that the authors analysed the data through several rounds, seeking consensus to establish codes (Cascio et al., 2019) and the older sentence “In addition, the authors analysed the data through several rounds, seeking consensus among the established codes” (lines 445 -446) will be deleted.
This study is exploratory in nature, thus we encourage researchers to delve deeper into how videogames can enhance DRM. Further research could improve our understanding of how specific narrative and gameplay elements (e.g. collaborative or competitive, duration of game and feedback and rewards systems), mechanics (e.g., mission achievement, creating new resources, discovering clues), and characters (e.g. different player roles, character characteristics, selectable avatars) in disaster-related video games improve DRM education. In this sense, subsequent studies could concentrate on validating the effectiveness of the proposed features in enhancing behaviour change of citizens for improving adaptation and resilience to natural hazards”.
Appendix:
Table B3 – Disaster Master/global history: rearrange this paragraph, moving the last sentence to the front (“the overarching framework …”).
**We thank you for this comment. We will rearrange this paragraph according to your suggestion.
Table B3 – Earthquake Relief Rescue+/Representation of character: cut out “but in a very impersonal manner” – avoid evaluating language.
**We will remove “but in a very impersonal manner”.
Table B6 – Competencies: include a footnote explaining the abbreviations.
**We appreciate the suggestion; however, a footnote explaining the abbreviations was included.
Table B6 – Learning curve: consider replacing “learning curve” with “level of difficulty”.
*We will replace “learning curve” with “level of difficulty”.
Table B6 – Accessibility: Explain what is meant by accessibility.
*We thank you for this comment. We will explain that accessibility refers to possibilities for people with functional diversity.
References
Cascio, M.A, Lee E., Vaudrin, N., Freedman, D. A.: A Team-based Approach to Open Coding: Considerations for Creating Intercoder Consensus, Field Methods, 31(2), 116-130, 2019.
Gampell, A.V., Gaillard, J.V.: Stop Disasters 2.0: Video Games as Tools for Disaster Risk Reduction, Int. J. Mass Emerg. Disasters, 34(2), 283-316, https://doi.org/10.1177/028072701603400205, 2016.
Ouariachi, T.; Gutiérrez-Pérez, J.; Olvera-Lobo, M.D: Criterios de evaluación de juegos online sobre cambio climático: aplicación del método Delphi para su identificación. Rev. Mex. Investig. Educ., 22, 445–474, 2017a.
Ouariachi, T.; Gutiérrez-Pérez, J.; Olvera-Lobo, M.D: Analyzing Climate Change Communication Through Online Games: Development and Application of Validated Criteria, Science Communication, 39(1) 10–44, 2017b.
Pérez-Latorre, O.: Análisis de la significación del videojuego: Fundamentos teóricos del juego, el mundo narrativo y la enunciación interactiva como perspectivas de estudio del discurso (Analisis of video game signification: game theoretic fundamentals, the narrative world and the interactive enunciation as perspectives for the discourse study) (Unpublished doctoral dissertation). Barcelona, Spain: Universitat Pompeu Fabra, Departament de Comunicació, 2019.
Safran, E.B., Nilsen, E., Drake, P., Sebok, B.: Effects of video game play, avatar choice, and avatar power on motivation to prepare for earthquakes, Int. J. Disaster Risk Reduct., 101, 104184, https://doi.org/10.1016/j.ijdrr.2023.104184, 2024.
Tanes, Z., Cho,H.: Goal setting outcomes: examining the role of goal interaction in influencing the experience and learning outcomes of video game play for earthquake preparedness, Comput. Hum. Behav. 29 (3), 858–869, 2013, https://doi.org/10.1016/j.chb.2012.11.003.
Tanes, Z.: Shall we play again? The effects of repetitive gameplay and self-efficacy on behavioural intentions to take earthquake precautions, Behav. Inf. Technol, 36 (10), 1037–45, 2017.
Weyrich, P., Ruin, I., Terti, G., Scolobig, A.: Using serious games to evaluate the potential of social media information in early warning disaster management, Int. J. Disaster Risk Reduct., 56, 102053, https://doi.org/10.1016/j.ijdrr.2021.102053, 2021.
-
AC2: 'Reply on RC2', Mercedes Vázquez-Vílchez, 19 Sep 2024
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-994', Miguel Angel Trejo Rangel, 22 Apr 2024
Dear authors,
Thank you for putting together this piece of paper. I found the topic very interesting and has large potential to be investigated to contribute to the DRM field. It is definitely an interesting topic to research considering that more and more serious games are emerging to enhance DRM.
I have some comments you can consider to improve the manuscript since I believe it can be publishable if adjustments are considered. I will go by section to make my suggestions clear enough.
The abstract and introduction sections are clear and well-presented, but I would suggest that in the introduction you expand on any other participatory approaches that have been reported in the literature to better frame why the research team decided to focus on serious gaming. As it is framed so far you exclude other approaches which are also important such as participatory mapping, workshops, hackathons or any other in-person and online methods to engage different stakeholders.
I liked how you explained serious gamin and their importance in the DRR in the theoretical framework, I think that is important considering that they have been used for some time already, however, you put together good references about them.
In the material and methods section, the section starts mentioning that it is qualitative research, however, when the results are presented, there are quantitative findings based on the questionnaires that were applied. I rather find this research as a mixed-method approach that complements findings from different methods.
For the content analysis, it is mentioned that the authors designed an instrument of analysis and evaluation. However, the results mostly focus on analysis and it is not very clear to me how it complements the focus group approach which for me should be a “focus group questionnaire”.
For the focus group (questionnaire), it is also not very clear how the MAXQDA software was used for analysing this method. I would expect the platform to be used in the content analysis section and not here. I know it is mentioned in the results, but I missed the implementation of a different method in the focus group section to gather participants' inputs. For instance, interviews or even an online workshop or so. I am concerned about conducting a questionnaire with that number of participants (8 video game experts and 14 natural hazard experts), which was not clear until I went through the results and figured it out.
I recognised that was very interesting to see how you selected the participants, but would be interesting to better understand how you approached them to answer the questionnaire.
In the results section, it is not a very friendly reader to keep mentioning tables that are pages ahead of the point you are reading. It is time-consuming and you get lost while you keep going to the table and coming back to the point where you stop reading. I strongly suggest that you try to condense the results of tables B1 to B6 into one figure. I am imagining a general description of these sections, followed by a figure, design, scheme, or something creative to picture what was found, and if the reader wants to get more details, can go to the detailed tables. Please allow yourself to think about how you would like to see that in a figure that represents the important information you have gathered already. I find this section very disconnected, and it does not give an overall picture of what was found. Also, I suggest to bring the discussions to the results section. Then, results can be compared to what has been reported in the literature. Then, you can discuss the type of related hazard, and lack of inclusivity due to the predominant languages (especially English) even when the public which could benefit from using these games does not use them, and other aspects.
Regarding the focus group results, I would try to merge both groups that participated and bring only key elements that contribute to answering your question “What are the educational and communicative elements or characteristics that serious games should have to improve DRM?”. You may notice that not all the answers from the questionnaire provide key information and you may need to prioritise what helps to respond to the question. If you decide to present the previous section (content analysis) on how serious games are in a more condensed way, you could take advantage of that and highlight how they “should improve” with these findings. I can see that the data collected in the content analysis could help to build a current picture and the information collected in questionnaires would be the transformed version of that.
Some charts and tables look a bit unnecessary since they do not say so much and take up a large space in the document. Please reconsider which of them are useful and how would be a better way to present more connected results where discussion can be included. I do see that you have collected valuable information, I just do not get how the results are organised, I believe this section should be better presented.
I suggest leaving the conclusions as a separate section without merging it with the discussions since I see a lot more potential to include discussions in the results section.
Please let me know if something is unclear, I am happy to clarify as much as possible.
Best regards,
The reviewer.
Citation: https://doi.org/10.5194/egusphere-2024-994-RC1 -
AC1: 'Reply on RC1', Mercedes Vázquez-Vílchez, 19 Sep 2024
Thank you for your constructive comments and suggestions. We believe addressing these comments will strengthen the paper and improve the message and key points we are trying to convey. Below, we respond to the specific comments, point by point and provide clarifications where necessary. We are confident that through this process we can improve the structure and effectiveness of the paper and communicate the results and conclusions more clearly.
Sincerely,
Dr Mercedes Vázquez Vílchez (on behalf of all co-authors)
Thank you for putting together this piece of paper. I found the topic very interesting and has large potential to be investigated to contribute to the DRM field. It is definitely an interesting topic to research considering that more and more serious games are emerging to enhance DRM.
I have some comments you can consider to improve the manuscript since I believe it can be publishable if adjustments are considered. I will go by section to make my suggestions clear enough.
** Thank you for this positive evaluation of our manuscript.
The abstract and introduction sections are clear and well-presented, but I would suggest that in the introduction you expand on any other participatory approaches that have been reported in the literature to better frame why the research team decided to focus on serious gaming. As it is framed so far you exclude other approaches which are also important such as participatory mapping, workshops, hackathons or any other in-person and online methods to engage different stakeholders.
** In the introduction, we have expanded on other participatory approaches and we clarify because we focus on serious gaming. We have introduced some sentences in the following introduction paragraph:
“Some authors suggested mutual learning, in order to promote the democratization of decisions, which combines a diverse learning methodology, such as adaptive management, experiential learning, or transformative learning (e.g. Lavell et al., 2012). Some important approaches in adaptive management incorporate the use of knowledge co-production, where scientists, politicians and other stakeholders work to interexchange, create and implement knowledge (van Kerkhoff and Lebel, 2006). In this sense, participatory mapping, workshop and hackathons are highlighted (e.g. Sullivan-Wiley et al., 2019; Trejo-Rangel et al., 2023; Macholl et al., 2023). These approaches introduce local knowledge of natural hazards into vulnerability evaluation, showing diverse vulnerabilities to natural hazards that are co-produced at local scales (Sullivan-Wiley et al., 2019). Experiential (Kolb, 1984) and transformative (Mezirow, 1995) learning remark the importance of action oriented to problem-solving, learning by-doing and how these processes create reflective thinking, theory generation and applications of knowledge, enabling behaviour change for adaptation to natural hazards (Sharpe, 2016; Lavell et al., 2012).
Following this approach, in which acquiring knowledge about natural hazards should enable citizens to make decisions and implement prevention measures, where there is recognition of active teaching methodologies, such as serious games, which may serve as a participatory and supportive tool for understanding the essential aspects of natural hazards (e.g. Solinska-Nowak et al., 2018, Tanwattana and Toyoda, 2018, Tsai et al., 2020; Schueller et al., 2020; Teague et al., 2021; Altan, et al., 2022; Villagra et al., 2023). Serious games allow users to visualise and explore phenomena that would otherwise be very difficult to experience as they enhance player immersion, and allow them to learn about the consequences of their actions at different points in time during a natural hazard (Solinska-Nowak et al., 2018; Heinzlef et al., 2024). In this way, serious games encourage experiential and transformative learning, as users try to reproduce a context as close to reality as possible that could allow to the players to enable behaviour change for adaptation and resilience to natural hazards (Villagra, 2023). The effectiveness of learning through serious games are also the immediate feedback and the emotional and sensorial experiences they provide, which is essential for learning to mitigate the effects of natural hazards (Solinska-Nowak et al., 2018; Heinzlef et al., 2024)”.
I liked how you explained serious gamin and their importance in the DRR in the theoretical framework, I think that is important considering that they have been used for some time already, however, you put together good references about them.
** Thank you for your kind words.
In the material and methods section, the section starts mentioning that it is qualitative research, however, when the results are presented, there are quantitative findings based on the questionnaires that were applied. I rather find this research as a mixed-method approach that complements findings from different methods.
** The approach to this study is qualitative. To avoid confusion, we will replace the word “questionnaire” with “semi-structured interviews”.
For the content analysis, it is mentioned that the authors designed an instrument of analysis and evaluation. However, the results mostly focus on analysis and it is not very clear to me how it complements the focus group approach which for me should be a “focus group questionnaire.
**Thank you for this comment. We will clarify this point. We will explain the content analysis in the text with the following paragraph and the Table B1 (Appendix) will appear in the main text as Table 1:
“A content analysis of the selected games was then carried out, which is a research tool used for the quantifying and analysis of the presence, meaning and relationship between specific words, themes or concepts, and thus inferences to be made about the messages within the different analysis unit (e.g. websites, journals, games, etc). The type of content analysis in this study is conceptual analysis. In conceptual content analysis a concept is chosen for examination and the analysis involves quantifying and counting its presence. It is able to identify the intentions or communicative tendencies in the games; in turn, it describes the attitudes or behaviours that result from those communications, revealing patterns in communication content. The dimensions analysed in this study were those proposed by Ouariachi et al. (2017a). These authors adapted the theoretical Social Discourse of Video Games Analysis Model (Pérez-Latorre, 2010) in an analysis instrument for games about climate change through the Delphi method. The instrument presents 51 criteria or variables, which are analysed in regards to the messages within the texts, audio, static and dynamic images of games. These criteria were classified into five dimensions: identification (features that help identify and locate the game), gameplay (set of properties that describe the player’s experience within a given game system), narrative (discursive construction around a complex phenomenon), contents (analysis of the information and messages transmitted) and educational aspects (referring to competencies, skills and learning). These criteria are described in further detail in Table 1. The analysis of the games was carried out by the authors, who played the games and filled out a form containing the criteria mentioned above.”
For the focus group (questionnaire), it is also not very clear how the MAXQDA software was used for analysing this method. I would expect the platform to be used in the content analysis section and not here. I know it is mentioned in the results, but I missed the implementation of a different method in the focus group section to gather participants' inputs. For instance, interviews or even an online workshop or so. I am concerned about conducting a questionnaire with that number of participants (8 video game experts and 14 natural hazard experts), which was not clear until I went through the results and figured it out.
**Thank you for this comment. We will clarify in the text that the semi-structured discussions were conducted online though open-ended questions. Two online semi-structured interviews were created using Google Forms, one addressed to natural hazards experts and earth science educators, and one addressed to video game experts. We will add the questions of the semi-structured interviews as supplementary materials (Tables S2 and S3). Once all the responses had been collected, codes were formulated based on their analysis using the program MAXQDA (2020) because this software allowed the authors to analyse the qualitative data collaboratively, create a common language in our codebook and reach consensus while benefiting from the unique perspectives of each team member.
I recognised that was very interesting to see how you selected the participants, but would be interesting to better understand how you approached them to answer the questionnaire.
**We will introduce details about how we approached to experts. In the text we will explain that the survey was carried out from March 10 to May 20, 2022. After sharing the interviews online, the experts in natural hazards and educators in earth sciences (13 experts) responded during the first week. However, it was necessary to insist with 42 video game experts to collect just 8 responses.
In the results section, it is not a very friendly reader to keep mentioning tables that are pages ahead of the point you are reading. It is time-consuming and you get lost while you keep going to the table and coming back to the point where you stop reading. I strongly suggest that you try to condense the results of tables B1 to B6 into one figure. I am imagining a general description of these sections, followed by a figure, design, scheme, or something creative to picture what was found, and if the reader wants to get more details, can go to the detailed tables. Please allow yourself to think about how you would like to see that in a figure that represents the important information you have gathered already. I find this section very disconnected, and it does not give an overall picture of what was found.
** We thank you for this comment. We will provide a figure (new Figure 1, see document attached) that condenses the information presented in Tables B1 to B6. Tables B1 to B6 will be included as supplementary material (Tables S4 to S8).
Also, I suggest to bring the discussions to the results section. Then, results can be compared to what has been reported in the literature. Then, you can discuss the type of related hazard, and lack of inclusivity due to the predominant languages (especially English) even when the public which could benefit from using these games does not use them, and other aspects.
** Based on your suggestion and that of the other reviewer, we will reorganize the text, and the paper will include separate sections for results, discussion, and conclusion.
Regarding the focus group results, I would try to merge both groups that participated and bring only key elements that contribute to answering your question “What are the educational and communicative elements or characteristics that serious games should have to improve DRM?”. You may notice that not all the answers from the questionnaire provide key information and you may need to prioritise what helps to respond to the question. If you decide to present the previous section (content analysis) on how serious games are in a more condensed way, you could take advantage of that and highlight how they “should improve” with these findings. I can see that the data collected in the content analysis could help to build a current picture and the information collected in questionnaires would be the transformed version of that.
**Thank you for this comment. We will eliminate those answers of the questionnaires that fail to provide key information for answering the research question “What are the educational and communicative elements or characteristics that serious games should have to improve DRM?”. We will delete the responses from videogame experts on curiosity and chance (line 291 to 298 in older version) and the corresponding older Figure 2. In this way, we will highlight those key answers directly related to the summary figure of content analysis of the games evaluated (new figure 1).
Some charts and tables look a bit unnecessary since they do not say so much and take up a large space in the document. Please reconsider which of them are useful and how would be a better way to present more connected results where discussion can be included. I do see that you have collected valuable information, I just do not get how the results are organised, I believe this section should be better presented.
**Thank you for this comment. We will reorganise the information presented. We will incorporate the key information of the older Figures 1, 3 and 4 in Tables 2 and 3 (categories of the expert responses) and, therefore, these older figures will be deleted.
I suggest leaving the conclusions as a separate section without merging it with the discussions since I see a lot more potential to include discussions in the results section.
**We now present the results, discussion and conclusions in separate sections in the paper.
References
Altan, B., Gürer, S., Alsamarei A., Kıvılcım Demir, D., Şebnem Düzgün, H., Erkayaoğlu, M.. Surer, E.: Developing serious games for CBRN-e training in mixed reality, virtual reality, and computer-based environments, Int. J. Disaster Risk Reduct., 77, 103022, https://doi.org/10.1016/j.ijdrr.2022.103022, 2022.
Heinzlef, C., Lamaury, Y., Serre, D.: Improving climate change resilience knowledge through a gaming approach: Application to marine submersion in the city of Punaauia, Tahiti, Environmental Advances, 15, 100467, 2024, 10.1016/j.envadv.2023.100467. hal-04353914.
Kolb, D.A.: Experiential Learning: Experience as the Source of Learning and Development. Prentice-Hall, Englewood Cliffs, NJ., 1984.
Lavell, A., Oppenheimer, M., Diop, C., Hess, J., Lempert, R., Li, J., Muir-Wood, R., Myeong, S., Moser, S., Takeuchi, K., Cardona, O. D., Hallegatte, S., Lemos, M., Little, C., Lotsch, A., and Weber, E.: Climate change: new dimensions in disaster risk, exposure, vulnerability, and resilience, in: Managing the risks of extreme events and disasters to advance climate change adaptation. A special report of working groups I and II of the Intergovernmental Panel on Climate Change (IPCC), edited by: Field, C.B., Barros, V., Stocker, T.F., Qin, D., Dokken, D.J., Ebi, K.L., Mastrandrea, M.D., Mach, K.J., Plattner, G.K., Allen, S.K., Tignor, M., Midgley P.M., Cambridge University Press, Cambridge and New York, 25–64, 2012.
Macholl, J.D., Roberts, H., Steptoe, H, Sun, S., Angus, M., Davenport, C., Luscombe, W., Rolker, H.B., Pope, E.C.D., Dawkins, L.C., Munday, G., Giles, D., Lam, T., Deutloff, J., Champion, A.J., Bloomfield, H.C., Mendes, J., Speight, L., Bradshaw, C.D., Wyatt, F. : A collaborative hackathon to investigate climate change and extreme weather impacts in justice and insurance settings, Weather, 79(6), 196-203., 2024.
Mezirow, J.: Transformation theory in adult learning. In: Defense of the Life World [Welton, M.R. (ed.)]. State University of New York Press, Albany, NY, 39-70, 1995.
Ouariachi, T.; Gutiérrez-Pérez, J.; Olvera-Lobo, M.D: Criterios de evaluación de juegos online sobre cambio climático: aplicación del método Delphi para su identificación. Rev. Mex. Investig. Educ., 22, 445–474, 2017a.
Pérez-Latorre, O.: Análisis de la significación del videojuego: Fundamentos teóricos del juego, el mundo narrativo y la enunciación interactiva como perspectivas de estudio del discurso (Analisis of video game signification: game theoretic fundamentals, the narrative world and the interactive enunciation as perspectives for the discourse study) (Unpublished doctoral dissertation). Barcelona, Spain: Universitat Pompeu Fabra, Departament de Comunicació, 2019.
Schueller, L., Booth, L., Fleming, K., Abad, J.: Using serious gaming to explore how uncertainty affects stakeholder decision-making across the science-policy divide during disasters, Int. J. Disaster Risk Reduct., 51, 101802, https://doi.org/10.1016/j.ijdrr.2020.101802, 2020.
Sharpe, J: Understanding and unlocking transformative learning as a method for enabling behaviour change for adaptation and resilience to disaster threats, Int. J. Disaster Risk Reduct, 17, 213-219, 2023.
Solinska-Nowak, A., Magnuszewski, P., Curl, M., French, A., Keating, A., Mochizuki, J., Liu, W., Mechler, R., Kulakowska, M., Jarzabek, L.: An overview of serious games for disaster risk management – prospects and limitations for informing actions to arrest increasing risk, Int. J. Disaster Risk Reduct., 31, 1013–1029, https://doi.org/10.1016/j.ijdrr.2018.09.001, 2018.
Sullivan-Wileya K.A., Short-Gianottib, A.G., Casellas Connorsc, J.P.: Mapping vulnerability: Opportunities and limitations of participatory community mapping, Applied Geography, 105: 47–57, 2019.
Tanwattana, P., Toyoda, Y.: Contributions of gaming simulation in building community-based disaster risk management applying Japanese case to flood prone communities in Thailand upstream area, Int. J. Disaster Risk Reduct., 27, 199-213, https://doi.org/10.1016/j.ijdrr.2017.10.007, 2018.
Teague, A., Sermet, Y., Demir, I., Muste, M.: A collaborative serious game for water resources planning and hazard mitigation, Int. J. Disaster Risk Reduct., 53, 101977, https://doi.org/10.1016/j.ijdrr.2020.101977, 2021.
Trejo-Rangel, M.A, Marchezini, V., Rodriguez, D.A., Messias dos Santos, D., Gabos, M, Lélis de Paula, A, Santos, E., Sampaio do Amaral, F.: Incorporating social innovations in the elaboration of disaster risk mitigation policies, Int. J. Disaster Risk Reduct, 84, 103450, 2023.
Tsai, M.H., Chang, Y.L., Shiau, J.S., Wang, S.M.: Exploring the effects of a serious game-based learning package for disaster prevention education: The case of Battle of Flooding Protection, Int. J. Disaster Risk Reduct., 43, 101393, https://doi.org/10.1016/j.ijdrr.2019.101393, 2020.
van Kerkhoff, L., L. Lebel: Linking knowledge and action for sustainable development. Annual Review of Environment and Resources, 31,445-477, 2006.
Villagra, P., Peña y Lillo, O., Ariccio, S., Bonaiuto, M. and Olivares-Rodríguez, C.: Effect of the Costa Resiliente serious game on community disaster resilience, Int. J. Disaster Risk Reduct., 91, 103686, https://doi.org/10.1016/j.ijdrr.2023.103686, 2023.
-
AC1: 'Reply on RC1', Mercedes Vázquez-Vílchez, 19 Sep 2024
-
RC2: 'Comment on egusphere-2024-994', Anonymous Referee #2, 08 Jul 2024
General comments
This paper examines the potential benefits of serious games for enhancing disaster risk management. The researchers employed a methodological triangulation approach, which involved content analysis, focus group discussions with experts and literature review. The findings show that online games are more successful in meeting experts' narrative criteria and have higher educational value, while mobile apps focus more on interaction, increasing engagement but lacking educational depth. The study also highlights a lack of attention to multiculturalism, diversity, and gender issues.
Thank you for your manuscript. The research questions posed are interesting and the research, per se, is relevant to the research area. However, the paper’s structure and the presentation of the information lack clarity and the grammar used is not at a level for publication.
- The grammar in the paper is seriously lacking throughout major parts of the paper. Therefore, please have the paper be proofread by a native speaker.
- The introduction would benefit from a clearer delineation of the research gap(s) and how this paper specifically is trying to answer it/them.
- I would appreciate it if the research questions was mentioned in the introduction and then specifically taken up again in the discussion part and answered in more detail
- Explain in more detail how the information in the results section were obtained. Adding the questionnaire to the appendix could be helpful to remedy that and help readers understand what the questionnaire looked like.
- consider if all of the figures presented in the results section are actually necessary or if the information presented in them could be condensed into one or two figure(s). Also rethink how the information in the tables is presented (see specific comment below)
- Some parts of the results section rather belongs to the discussion and conclusion chapter, i.e. leave all interpretation of the results to the discussion. Some, but by far not all, of the instances are commented on in the specific comments (see specific comment below)
- I would separate the discussion and conclusion section so as to make it more reader-friendly
- the paper could benefit from a clearer use of terminology = unify the use of i.e. natural risks, natural hazards, natural hazards-related disaster
- refrain from using evaluating language if it is not language used by the experts (and if the latter, use direct quotes)
Specific comments:
Section 2 (Theoretical framework):
Line 99: add percentage for the prevalence of storms
Section 3 (Materials and methods):
Line 135: the age range is unclear – if the game is geared at young people without giving an age upper limit, it should rather be argued that it is for young and adult people
Line 137: it would be helpful for the reader to know on the basis of which criteria this selection was made
Line 140ff: describe in more detail what the content analysis looked like with games as objects of the study. How was content extracted from the games, which aspects of the structure of the games were analyzed (i.e. visual, audio, text, etc.) and how. How did you deal with the interactive aspect of the games, i.e. different outcomes of the games depending on the players’ decisions, etc.?
Line 144: “First” is not needed if there are no other arguments listed (“second”, “third”)
Line 147: fix reference (only year in brackets)
Line 152: close bracket
Lines 163-165: Rewrite accordingly: This study involved individuals with at least 5 years of professional or experiential knowledge of the research topic, constituting an informed panel and thereby justifying the use of the title "experts" (Mullen, 2003).
Line 170: include an explanation of how these participants are experts in natural hazards, i.e. are they researchers, practitioners, stakeholders…
Line 179: add citation for MAXQDA program. Also, it is unclear what the program was used for.
Section 4 – Results:
Clarify how this information was obtained. Were the following questions posed as open questions, was there a selection of answers given and could experts add own ideas, …? As mentioned above, adding the appendix to the paper helps readers understand how this information was elicited. This applies to all other questions in the following tables in the result section.
Rethink the use of figures (see comment above) and consider condensing some figures into one. Also, the tables do not present information in a very reader-friendly way, consider rephrasing the items shown so it is more clear to the readers what they mean.
Line 188: To me, there seems to be no point in highlighting Hungarian and Turkish as available languages for the paper at hand, so delete and rather than pointing out specific languages maybe state the number of languages in which the game is available
Line 221: add why the term “natural risk” in particular is one you looked out for. Also I would, again, argue that this is content for the discussion section of the paper
Line 221-222: explain in more detail – why is this an interesting result and what constitutes a complex concept (consider also giving examples). Again, this would be content for the discussion section.
Line 231: Consider moving this whole subsection (4.1.4) to the beginning of the chapter. Painting a picture of the games’ setup could help readers in getting an understanding of the games and differentiating them better
Line 233: use “in multi-player mode” instead of “collective way”
Line 238: make the link clearer as to why the online games offer educational advantages over the mobile ones. What characterizes the mobile ones in comparison?
Line 243: Clarify what you mean by “the other games might also be played in an hour”
Line 245: Clarify what the difference is between positive feedback and a reward system
Line 250: Explain what Bloom’s Taxonomy is and how this is relevant here
Line 258f: Clarify if only online games dealt with these dimensions
Line 266ff/Table 1: clarify what “Always” means in “reward systems” section, especially since this is not referred to in the text. Also consider renaming this item as it is too unspecific
Line 266ff/Table 1: clarify why progress and motivation is handled as one category. Consider renaming the third category to “motivate” in order to make terminology uniform (i.e. all verb infinitives or the gerund). Do this for the entire table.
Line 266ff/Table 1: the last 3 subcategories in the “duration” section are confusing to me as their connection to the theme of duration only makes sense after reading the text. This, to me, raises the question if the information should rather be presented in an answer-like style, i.e. “game should not take more than an hour” “game should take up to several hours”, “game length can be variable if narrative is employed”, etc. Such rephrasing could help readers understand the subcategories better.
Line 280: 46% does not appear in the table. Check if numbers in text and table align.
Line 291: Figure 1 is not referred to in the text and, even if it were, seems unnecessary – information depicted does not need its own figure
Line 293: “All of the experts agree” does not fit to the number in the cited figure (figure only shows 62% for curiosity).
Lines 295ff: the last to sentences of the paragraph seem to repeat the same information.
Line 301: Unsure whether this conclusion can be drawn from merely 7% and 21% of responses.
Line 302f: This is an interpretation, move this part to the discussion section.
Table 2: there is a typo in “multiculturalis”
Table 2: change phrasing of “normal person” – avoid evaluative language if it is not language explicitly used by the experts. If that were the case, use direct quotes
Figure 5: figure is cut off on the left
Line 363f: cut “wonderful” and “great” - avoid evaluative language if it is not language explicitly used by the experts. If that were the case, use direct quotes
Section 5 (discussion and conclusion):
Move all interpretation of the obtained data currently found in the results section here and consider splitting discussion and conclusion into two separate chapters
Lines 402ff: rephrase this sentence as it is not clear which of the aspects named are those of “secondary presence”
Section 6 (limits and recommendation for future studies):
This section requires more detail and clarity, i.e. what aspects of the methodological approach of triangulation helps in preventing a subjective interpretation of the results? Clarify what you mean by “seeking consensus”. Give more detailed examples of aspects to be studies for future research.
Appendix:
Table B3 – Disaster Master/global history: rearrange this paragraph, moving the last sentence to the front (“the overarching framework …”)
Table B3 – Earthquake Relief Rescue+/Representation of character: cut out “but in a very impersonal manner” – avoid evaluating language
Table B6 – Competencies: include a footnote explaining the abbreviations
Table B6 – Learning curve: consider replacing “learning curve” with “level of difficulty”
Table B6 – Accessibility: Explain what is meant by accessibility
Citation: https://doi.org/10.5194/egusphere-2024-994-RC2 -
AC2: 'Reply on RC2', Mercedes Vázquez-Vílchez, 19 Sep 2024
Thank you for your constructive comments and suggestions. We believe addressing these comments will strengthen the paper and improve the message and key points we are trying to convey. Below, we respond to the specific comments, point by point and provide clarifications where necessary. We are confident that through this process we can improve the structure and effectiveness of the paper and communicate the results and conclusions more clearly.
Sincerely,
Dr Mercedes Vázquez Vílchez (on behalf of all co-authors)
This paper examines the potential benefits of serious games for enhancing disaster risk management. The researchers employed a methodological triangulation approach, which involved content analysis, focus group discussions with experts and literature review. The findings show that online games are more successful in meeting experts' narrative criteria and have higher educational value, while mobile apps focus more on interaction, increasing engagement but lacking educational depth. The study also highlights a lack of attention to multiculturalism, diversity, and gender issues.
Thank you for your manuscript. The research questions posed are interesting and the research, per se, is relevant to the research area. However, the paper’s structure and the presentation of the information lack clarity and the grammar used is not at a level for publication.
** We thank the reviewer for the careful review. We agree that the paper will greatly benefit from modifying the structure, improving the presentation of the information, and enhancing the grammar.
- The grammar in the paper is seriously lacking throughout major parts of the paper. Therefore, please have the paper be proofread by a native speaker.
** A native speaker will revise the paper.
- The introduction would benefit from a clearer delineation of the research gap(s) and how this paper specifically is trying to answer it/them.
**Thank you for this comment. We will clarify the research gaps in the introduction section. The text will include a new sentence in the following paragraph: “However, while serious games can contribute by giving researchers useful evidence into how people conceive disasters, there is a poor understanding of the representation of catastrophes within popular culture (Gampell and Gaillard, 2016; Safran et al., 2024). Some authors related the characteristics of several disaster games to the disaster risk reduction framework (mitigation, preparedness and recovery), highlighting the need for further research into how game characteristics (mechanics, dynamics, narrative and content), player skills, motivations and social interactions contribute towards improving decision-making in the area of disaster risk reduction (DRR) (Gampell and Gaillard, 2016; Safran et al., 2024). Few works address the influence of video games on the tendency of players to prepare for natural hazards (Tanes and Cho, 2013; Tanes, 2017; Safran et al, 2024). Attention is drawn to the lack of solid scientific evidence of the potential of serious games, with challenges remaining for the development of more detailed studies to test and demonstrate the effectiveness of serious games for DRM education (Weyrich et al., 2021; Safran et al., 2024).
- I would appreciate it if the research questions was mentioned in the introduction and then specifically taken up again in the discussion part and answered in more detail.
**Thank you for this comment. We will specify the research question in the introduction. The following paragraph will appear in the text: “This paper aims to explore the potential of serious games for improving DRM. The main research question it raises is: How can we educate and communicate more effectively about natural hazards through serious games? To this end, the following research sub-questions are posed: (a) How do serious games communicate and educate on issues related to natural hazards? and (b) Which educational and communicative elements or characteristics should serious games have to improve DRM? This work will follow a methodology consisting of triangulation-based qualitative research. The research method includes a content analysis of selected serious games applied to DRM, a focus group of experts and a literature review (the constructs of the investigation were determined with the help of introductory literature)”. Also we will indicate the research question in the discussion to provide a more detailed answer.
-Explain in more detail how the information in the results section were obtained. Adding the questionnaire to the appendix could be helpful to remedy that and help readers understand what the questionnaire looked like.
**The questions of the semi-structured interviews will be added as supplementary materials (Table S2 and S3).
-consider if all of the figures presented in the results section are actually necessary or if the information presented in them could be condensed into one or two figure(s). Also rethink how the information in the tables is presented (see specific comment below).
**We agree that not all figures presented are necessary. We will incorporate the key information from Figures 1, 3, and 4 into Tables 2 and 3 (expert response categories) and, therefore, these figures will be deleted. Figure 2 will also be eliminated based on feedback from another reviewer.
-Some parts of the results section rather belongs to the discussion and conclusion chapter, i.e. leave all interpretation of the results to the discussion. Some, but by far not all, of the instances are commented on in the specific comments (see specific comment below).
**Thank you for this comment. The interpretations will be moved from the results to the discussion and conclusion sections.
- I would separate the discussion and conclusion section so as to make it more reader-friendly.
**We appreciate the suggestion and will separate the discussion and conclusions.
- the paper could benefit from a clearer use of terminology = unify the use of i.e. natural risks, natural hazards, natural hazards-related disaster.
** We will unify the terminology using only natural hazards.
- refrain from using evaluating language if it is not language used by the experts (and if the latter, use direct quotes).
**We thank you for this comment. We use the language used by the experts. However, we acknowledge the potential confusion and we will add two tables with responses of the experts as supplementary material (Tables S9 and S10, see attached document).
Specific comments:
Section 2 (Theoretical framework):
Line 99: add percentage for the prevalence of storms.
**We will add 34% as a percentage for the prevalence of storms.
Section 3 (Materials and methods):
Line 135: the age range is unclear – if the game is geared at young people without giving an age upper limit, it should rather be argued that it is for young and adult people.
**We will clarify that the games evaluated are aimed at young people and adults (age 12 and upwards).
Line 137: it would be helpful for the reader to know on the basis of which criteria this selection was made.
**We will remark that according to the criteria explained previously, in this phase, from 17 digital games, a total of 6 games were selected: 3 mobile apps (Earthquake Relief Rescue, Geostorm and Disaster Rescue Service) and 3 online games (Build a kit, Disaster Master and Stop Disasters).
Line 140ff: describe in more detail what the content analysis looked like with games as objects of the study. How was content extracted from the games, which aspects of the structure of the games were analyzed (i.e. visual, audio, text, etc.) and how. How did you deal with the interactive aspect of the games, i.e. different outcomes of the games depending on the players’ decisions, etc.?
** Thank you for this comment. We will clarify this point. We will explain the content analysis in the text with the following paragraph and the Table B1 (Appendix) will appear in the main text as Table 1:
“A content analysis of the selected games was then carried out, which is a research tool used for the quantifying and analysis of the presence, meaning and relationship between specific words, themes or concepts, and thus inferences to be made about the messages within the different analysis unit (e.g. websites, journals, games, etc). The type of content analysis in this study is conceptual analysis. In conceptual content analysis a concept is chosen for examination and the analysis involves quantifying and counting its presence. It is able to identify the intentions or communicative tendencies in the games; in turn, it describes the attitudes or behaviours that result from those communications, revealing patterns in communication content. The dimensions analysed in this study were those proposed by Ouariachi et al. (2017a). These authors adapted the theoretical Social Discourse of Video Games Analysis Model (Pérez-Latorre, 2010) in an analysis instrument for games about climate change through the Delphi method. The instrument presents 51 criteria or variables, which are analysed in regards to the messages within the texts, audio, static and dynamic images of games. These criteria were classified into five dimensions: identification (features that help identify and locate the game), gameplay (set of properties that describe the player’s experience within a given game system), narrative (discursive construction around a complex phenomenon), contents (analysis of the information and messages transmitted) and educational aspects (referring to competencies, skills and learning). These criteria are described in further detail in Table 1. The analysis of the games was carried out by the authors, who played the games and filled out a form containing the criteria mentioned above.”
Line 144: “First” is not needed if there are no other arguments listed (“second”, “third”).
**We thank you for this comment. We will modify this paragraph and the first word will be deleted.
Line 147: fix reference (only year in brackets).
**We will correct this reference.
Line 152: close bracket.
** We will close the brackets.
Lines 163-165: Rewrite accordingly: This study involved individuals with at least 5 years of professional or experiential knowledge of the research topic, constituting an informed panel and thereby justifying the use of the title "experts" (Mullen, 2003).
**Thank you for this comment. We will properly rewrite this sentence.
Line 170: include an explanation of how these participants are experts in natural hazards, i.e. are they researchers, practitioners, stakeholders…
**We will add the following explanation to the paper: As regards the surveys of experts in natural hazards, they present different natural hazard backgrounds (climate change, floods, earthquakes, volcanoes and mass movement) and professions (researchers, emergencies experts, a politician and the partner of a consulting company in urban and territorial planning).
Line 179: add citation for MAXQDA program. Also, it is unclear what the program was used for.
**Thank you for this comment. We will include the correct reference; MAXQDA (2020) will appear in the text and we will explain that this qualitative data analysis software allowed the authors to analyse the data collaboratively, create a common language in our codebook and reach consensus while benefiting from the unique perspectives of each team member.
Section 4 – Results:
Clarify how this information was obtained. Were the following questions posed as open questions, was there a selection of answers given and could experts add own ideas, …? As mentioned above, adding the appendix to the paper helps readers understand how this information was elicited. This applies to all other questions in the following tables in the result section.
**Thank you for this comment. The semi-structured discussions were conducted online though open-ended questions. We will add the questions of these as supplementary materials (Table S2 and S3).
Rethink the use of figures (see comment above) and consider condensing some figures into one. Also, the tables do not present information in a very reader-friendly way, consider rephrasing the items shown so it is more clear to the readers what they mean.
**We will incorporate the key information from Figures 1, 3, and 4 into Tables 2 and 3 (expert response categories) and, therefore, these figures will be deleted. Figure 2 will also be eliminated based on feedback from another reviewer. We will provide a figure (new Figure1, see document attached) that condenses the information presented in Tables B1 to B6. Tables from B1 to B6 will be included as supplementary material (Tables S4 to S8). In order to improve the understanding of the tables some categories will be rephrased.
Line 188: To me, there seems to be no point in highlighting Hungarian and Turkish as available languages for the paper at hand, so delete and rather than pointing out specific languages maybe state the number of languages in which the game is available.
**We will specify that the Geostorm game gives the option of 11 languages.
Line 221: add why the term “natural risk” in particular is one you looked out for. Also I would, again, argue that this is content for the discussion section of the paper.
*We thank you for this comment. We will delete the following sentence: “We did not find the term "natural risk", but rather "natural catastrophe" or "natural disaster", because in the text it is enough clear that the games use rather alarmist terms such as "emergency", "catastrophe" and "disaster”.
Line 221-222: explain in more detail – why is this an interesting result and what constitutes a complex concept (consider also giving examples). Again, this would be content for the discussion section.
**We agree, and this sentence will be moved to discussion section. We will add examples of complex concepts such as earthquake epicentre, floodplain, etc.
Line 231: Consider moving this whole subsection (4.1.4) to the beginning of the chapter. Painting a picture of the games’ setup could help readers in getting an understanding of the games and differentiating them better.
**We will relocate the gameplay subsection to the beginning of the chapter. We will provide a figure (new Figure1, see attached document) that condenses the information presented from Tables B1 to B6, including Table B5 (game play dimension).
Line 233: use “in multi-player mode” instead of “collective way”.
**We agree, we will use multi-player mode.
Line 238: make the link clearer as to why the online games offer educational advantages over the mobile ones. What characterizes the mobile ones in comparison?.
**The sentence: “Therefore, these three online games offer further educational advantages over the mobile app examples”will be deleted. This idea will be integrated into the discussion section.
Line 243: Clarify what you mean by “the other games might also be played in an hour”.
**Thank you for this comment. The sentence will be modified as: “The other games present levels that might also be played in one hour”.
Line 245: Clarify what the difference is between positive feedback and a reward system.
** We will clarify this point through the following addition to the text : “Positive feedback, though messages the player receives in light of certain actions, are abundant in the games, and we only found a reward system in the online examples”. In addition, this point is extensively explained in Table B1 (Table1 in new version).
Line 250: Explain what Bloom’s Taxonomy is and how this is relevant here.
**Thank you for this comment. We will explain that Bloom's Taxonomy consists of a hierarchical structure of objectives or levels that allow educators to evaluate the learning process of students; it is also a useful starting point for designing activities to achieve meaningful and lifelong learning. Accordingly, the evaluation criteria related to "Remember" and "Understand" are classified as "Basic"; the criteria related to "Apply" and "Analyse" are catalogued as "Optimal"; and the criteria related to "Evaluate" and "Create" are classified as "Desirable". Taking into account these levels, the most complete are Stop Disasters, Disaster Master and Geostorm which cover all of them; however, Earthquake Relief Rescue and Disaster Rescue Service are the two games that cover only the Basic and Optimal learning levels."
Line 258f: Clarify if only online games dealt with these dimensions.
**We will clarify that the Multiculturalism and Cultural Awareness and Expression Competences are addressed only by the online games (Build a kit, Disaster Master and Stop Disasters).
Line 266ff/Table 1: clarify what “Always” means in “reward systems” section, especially since this is not referred to in the text. Also consider renaming this item as it is too unspecific.
**Based on the other reviewer, this category will be eliminated from Table 1 (Table 2 in revised version), because it does not contribute towards answering the research question. In addition, the idea expressed by the experts in the recommended/satisfactory category is too similar to the Always category.
Line 266ff/Table 1: clarify why progress and motivation is handled as one category. Consider renaming the third category to “motivate” in order to make terminology uniform (i.e. all verb infinitives or the gerund). Do this for the entire table.
**According to the suggestion of the other reviewer, these categories from the dimension level will be eliminated from Table 1 (Table 2 in the new version), as they do not contribute towards answering the research question.
Line 266ff/Table 1: the last 3 subcategories in the “duration” section are confusing to me as their connection to the theme of duration only makes sense after reading the text. This, to me, raises the question if the information should rather be presented in an answer-like style, i.e. “game should not take more than an hour” “game should take up to several hours”, “game length can be variable if narrative is employed”, etc. Such rephrasing could help readers understand the subcategories better.
**Based on the suggestion of the other reviewer, the categories 'narrative' and 'engage' will be eliminated from Table 1 (Table 2 in new version) as they do not contribute towards answering the research question. We will add new tables with examples of expert responses in the supplementary materials to improve the understanding of the remaining categories (Tables S9 and S10, see attached documents).
Line 280: 46% does not appear in the table. Check if numbers in text and table align.
**We thank you for this comment. We will correct the data.
Line 291: Figure 1 is not referred to in the text and, even if it were, seems unnecessary – information depicted does not need its own figure.
**Figure 1 will be deleted and the data will appear in Table 1 (Table 2 in the new version).
Line 293: “All of the experts agree” does not fit to the number in the cited figure (figure only shows 62% for curiosity).
**According to the other reviewer, we will eliminate those answers of the semi-structured interviews that do not provide key information towards answering the research question: What are the educational and communicative elements or characteristics that serious games should have to improve DRM? We will delete the responses from videogame experts on curiosity and chance (line 291 to 298) and we will delete the corresponding Figure 2. In this way, we will highlight those key answers directly related to the content analysis of the games evaluated.
Lines 295ff: the last to sentences of the paragraph seem to repeat the same information.
**As we mentioned in the previous point, we will delete the responses from videogame experts on curiosity and chance (line 291 to 298).
Line 301: Unsure whether this conclusion can be drawn from merely 7% and 21% of responses.
**We thank you for this comment. We will delete the categories fun and engage, corresponding to 7% and 21% of the level of interaction dimension. We will include the level of interaction dimension with only one category (narrative) in Table 1 (Table 2 in the new version).
Line 302f: This is an interpretation, move this part to the discussion section.
**We appreciate the suggestion and we will modify this sentence as: “However, the code most represented in this question is narrative, since in a serious game with strong narrative the interaction can be lower and they can have a great impact.” We will add two tables in supplementary materials showing the experts’ answers (see document attached).
Table 2: there is a typo in “multiculturalis”.
**We will correct this typo.
Table 2: change phrasing of “normal person” – avoid evaluative language if it is not language explicitly used by the experts. If that were the case, use direct quotes.
**We thank you for this comment. We do not use evaluative language; instead, we use the language explicitly employed by the experts. We will thoroughly review the results to relocate any sentences that include the authors' interpretations. We will add two tables with direct quotes from experts to help readers understand the results (Tables S9 and 10, see documents attached).
Figure 5: figure is cut off on the left.
**We will correct this in the revised version.
Line 363f: cut “wonderful” and “great” - avoid evaluative language if it is not language explicitly used by the experts. If that were the case, use direct quotes.
**We appreciate the suggestion and, in this case, we will delete these words from the text to avoid evaluative language.
Section 5 (discussion and conclusion):
Move all interpretation of the obtained data currently found in the results section here and consider splitting discussion and conclusion into two separate chapters.
**We will move all interpretations from the results section to a new discussion section. We will provide a separate section for the conclusion.
Lines 402ff: rephrase this sentence, as it is not clear which of the aspects named are those of “secondary presence”.
**We will rephrase this sentence. The new version will include: “These desirable characteristics are: exploratory characters in a cooperative dynamic, with simple and non-technical narratives, based on academic sources, multiculturalism, consideration of diversity and gender, fun, short games, constructive feedback system, while the rewards could have a secondary presence.”
Section 6 (limits and recommendation for future studies):
This section requires more detail and clarity, i.e. what aspects of the methodological approach of triangulation helps in preventing a subjective interpretation of the results? Clarify what you mean by “seeking consensus”. Give more detailed examples of aspects to be studies for future research.
** We will rewrite this section. In the new version of the paper it will appear as:
“The limitations of this study are related to the subjective nature of qualitative analyses, due to their focus on interpreting meaning and the meaning-making process. Consequently, a methodical triangulation approach (content analysis of selected serious games, focus group with experts and literature review) was used to minimize this limitation (Ouariachi et al., 2017b).
On the other hand, in section 3.2 Focus group with experts (3. Materials and methods) we will clarify that the authors analysed the data through several rounds, seeking consensus to establish codes (Cascio et al., 2019) and the older sentence “In addition, the authors analysed the data through several rounds, seeking consensus among the established codes” (lines 445 -446) will be deleted.
This study is exploratory in nature, thus we encourage researchers to delve deeper into how videogames can enhance DRM. Further research could improve our understanding of how specific narrative and gameplay elements (e.g. collaborative or competitive, duration of game and feedback and rewards systems), mechanics (e.g., mission achievement, creating new resources, discovering clues), and characters (e.g. different player roles, character characteristics, selectable avatars) in disaster-related video games improve DRM education. In this sense, subsequent studies could concentrate on validating the effectiveness of the proposed features in enhancing behaviour change of citizens for improving adaptation and resilience to natural hazards”.
Appendix:
Table B3 – Disaster Master/global history: rearrange this paragraph, moving the last sentence to the front (“the overarching framework …”).
**We thank you for this comment. We will rearrange this paragraph according to your suggestion.
Table B3 – Earthquake Relief Rescue+/Representation of character: cut out “but in a very impersonal manner” – avoid evaluating language.
**We will remove “but in a very impersonal manner”.
Table B6 – Competencies: include a footnote explaining the abbreviations.
**We appreciate the suggestion; however, a footnote explaining the abbreviations was included.
Table B6 – Learning curve: consider replacing “learning curve” with “level of difficulty”.
*We will replace “learning curve” with “level of difficulty”.
Table B6 – Accessibility: Explain what is meant by accessibility.
*We thank you for this comment. We will explain that accessibility refers to possibilities for people with functional diversity.
References
Cascio, M.A, Lee E., Vaudrin, N., Freedman, D. A.: A Team-based Approach to Open Coding: Considerations for Creating Intercoder Consensus, Field Methods, 31(2), 116-130, 2019.
Gampell, A.V., Gaillard, J.V.: Stop Disasters 2.0: Video Games as Tools for Disaster Risk Reduction, Int. J. Mass Emerg. Disasters, 34(2), 283-316, https://doi.org/10.1177/028072701603400205, 2016.
Ouariachi, T.; Gutiérrez-Pérez, J.; Olvera-Lobo, M.D: Criterios de evaluación de juegos online sobre cambio climático: aplicación del método Delphi para su identificación. Rev. Mex. Investig. Educ., 22, 445–474, 2017a.
Ouariachi, T.; Gutiérrez-Pérez, J.; Olvera-Lobo, M.D: Analyzing Climate Change Communication Through Online Games: Development and Application of Validated Criteria, Science Communication, 39(1) 10–44, 2017b.
Pérez-Latorre, O.: Análisis de la significación del videojuego: Fundamentos teóricos del juego, el mundo narrativo y la enunciación interactiva como perspectivas de estudio del discurso (Analisis of video game signification: game theoretic fundamentals, the narrative world and the interactive enunciation as perspectives for the discourse study) (Unpublished doctoral dissertation). Barcelona, Spain: Universitat Pompeu Fabra, Departament de Comunicació, 2019.
Safran, E.B., Nilsen, E., Drake, P., Sebok, B.: Effects of video game play, avatar choice, and avatar power on motivation to prepare for earthquakes, Int. J. Disaster Risk Reduct., 101, 104184, https://doi.org/10.1016/j.ijdrr.2023.104184, 2024.
Tanes, Z., Cho,H.: Goal setting outcomes: examining the role of goal interaction in influencing the experience and learning outcomes of video game play for earthquake preparedness, Comput. Hum. Behav. 29 (3), 858–869, 2013, https://doi.org/10.1016/j.chb.2012.11.003.
Tanes, Z.: Shall we play again? The effects of repetitive gameplay and self-efficacy on behavioural intentions to take earthquake precautions, Behav. Inf. Technol, 36 (10), 1037–45, 2017.
Weyrich, P., Ruin, I., Terti, G., Scolobig, A.: Using serious games to evaluate the potential of social media information in early warning disaster management, Int. J. Disaster Risk Reduct., 56, 102053, https://doi.org/10.1016/j.ijdrr.2021.102053, 2021.
-
AC2: 'Reply on RC2', Mercedes Vázquez-Vílchez, 19 Sep 2024
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 1,494 | 251 | 244 | 1,989 | 34 | 50 |
- HTML: 1,494
- PDF: 251
- XML: 244
- Total: 1,989
- BibTeX: 34
- EndNote: 50
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Cited
1 citations as recorded by crossref.
Mercedes Vázquez-Vílchez
Rocío Carmona-Molero
Tania Ouariachi-Peralta
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(928 KB) - Metadata XML
Dear authors,
Thank you for putting together this piece of paper. I found the topic very interesting and has large potential to be investigated to contribute to the DRM field. It is definitely an interesting topic to research considering that more and more serious games are emerging to enhance DRM.
I have some comments you can consider to improve the manuscript since I believe it can be publishable if adjustments are considered. I will go by section to make my suggestions clear enough.
The abstract and introduction sections are clear and well-presented, but I would suggest that in the introduction you expand on any other participatory approaches that have been reported in the literature to better frame why the research team decided to focus on serious gaming. As it is framed so far you exclude other approaches which are also important such as participatory mapping, workshops, hackathons or any other in-person and online methods to engage different stakeholders.
I liked how you explained serious gamin and their importance in the DRR in the theoretical framework, I think that is important considering that they have been used for some time already, however, you put together good references about them.
In the material and methods section, the section starts mentioning that it is qualitative research, however, when the results are presented, there are quantitative findings based on the questionnaires that were applied. I rather find this research as a mixed-method approach that complements findings from different methods.
For the content analysis, it is mentioned that the authors designed an instrument of analysis and evaluation. However, the results mostly focus on analysis and it is not very clear to me how it complements the focus group approach which for me should be a “focus group questionnaire”.
For the focus group (questionnaire), it is also not very clear how the MAXQDA software was used for analysing this method. I would expect the platform to be used in the content analysis section and not here. I know it is mentioned in the results, but I missed the implementation of a different method in the focus group section to gather participants' inputs. For instance, interviews or even an online workshop or so. I am concerned about conducting a questionnaire with that number of participants (8 video game experts and 14 natural hazard experts), which was not clear until I went through the results and figured it out.
I recognised that was very interesting to see how you selected the participants, but would be interesting to better understand how you approached them to answer the questionnaire.
In the results section, it is not a very friendly reader to keep mentioning tables that are pages ahead of the point you are reading. It is time-consuming and you get lost while you keep going to the table and coming back to the point where you stop reading. I strongly suggest that you try to condense the results of tables B1 to B6 into one figure. I am imagining a general description of these sections, followed by a figure, design, scheme, or something creative to picture what was found, and if the reader wants to get more details, can go to the detailed tables. Please allow yourself to think about how you would like to see that in a figure that represents the important information you have gathered already. I find this section very disconnected, and it does not give an overall picture of what was found. Also, I suggest to bring the discussions to the results section. Then, results can be compared to what has been reported in the literature. Then, you can discuss the type of related hazard, and lack of inclusivity due to the predominant languages (especially English) even when the public which could benefit from using these games does not use them, and other aspects.
Regarding the focus group results, I would try to merge both groups that participated and bring only key elements that contribute to answering your question “What are the educational and communicative elements or characteristics that serious games should have to improve DRM?”. You may notice that not all the answers from the questionnaire provide key information and you may need to prioritise what helps to respond to the question. If you decide to present the previous section (content analysis) on how serious games are in a more condensed way, you could take advantage of that and highlight how they “should improve” with these findings. I can see that the data collected in the content analysis could help to build a current picture and the information collected in questionnaires would be the transformed version of that.
Some charts and tables look a bit unnecessary since they do not say so much and take up a large space in the document. Please reconsider which of them are useful and how would be a better way to present more connected results where discussion can be included. I do see that you have collected valuable information, I just do not get how the results are organised, I believe this section should be better presented.
I suggest leaving the conclusions as a separate section without merging it with the discussions since I see a lot more potential to include discussions in the results section.
Please let me know if something is unclear, I am happy to clarify as much as possible.
Best regards,
The reviewer.