the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 22 Jan 2024
Minions of Disruptions™: A Collaborative Adaptation Game for Promoting Climate Action
Abstract. With the onset of climate change, adaptive action will have to occur at all scales, including locally. This implies a growing responsibility for the public and, therefore, a need to spread awareness and inspire climate action. Communication via adaptation games shows potential in achieving social learning and addressing the so-called knowledge-action gap. However, few research efforts so far give voice to participants engaging with collaborative games in organisational and community settings. This paper advances this field by presenting a systematic research reflection on a collaborative tabletop board game, Minions of Disruptions™. It addresses two research questions: first exploring how to design a collaborative adaptation game for the general public, and then determining how the intentions outlined by the game designers are perceived by the game participants. Ten core design intentions determined through a focus group interview with game designers and facilitators were contrasted against responses from the post-game survey administered to all game participants from 2019–2022. The results of this study indicate that the design intentions behind Minions of Disruptions were largely received by the intended audiences, demonstrating success as a communication tool for collaborative climate action. Moreover, important insights about designing adaptation games for the public are raised, which can aid in drafting guidelines for successful engagement.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(982 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(982 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-46', Pratama Atmaja, 27 Feb 2024
The manuscript presents a novel contribution to the important yet emerging body of knowledge of games for complex issue communication. Due to their highly interactive and engaging nature, games are increasingly employed to communicate climate crises and other complex issues to the public. One glaring question regarding this communication method is how to ensure that the message is well-received, and this manuscript offers a preliminary yet compelling answer. It begins with providing a concise overview of the current literature on climate crises and games for climate-related communication and education. It then presents a case study of a specific board game for promoting climate actions. It then shows that the game’s effectiveness can be measured by identifying the developer’s design intentions, design elements, and the players’ impressions of the intentions through a focus group and a survey. The manuscript discusses its research method, data, and results in great detail, culminating in design principles and directions for future research, the former useful for scholars and game developers wanting to replicate the game’s success.
At least that is what the manuscript seems to aim for. Unfortunately, its contributions are held back somewhat by ambiguities. At the core of its case study, the manuscript presents three findings: (1) the developer’s design intentions, (2) the players’ impressions of the design intentions, and (3) some design elements that are supposed to transmit the intentions to the players. The first ambiguity revolves around the first and third findings. Looking at the design elements’ descriptions, it is rather difficult to imagine exactly how some elements transmit some of the design intentions. For example, how exactly do “uncontrollable events” and “medium: board” transmit “relatability? The authors should elaborate on this more to make the findings more informative to other scholars and game developers. Additionally, the players’ impressions of the developer’s intentions are also riddled with ambiguity. For example, when some players complained about insufficient time for a discussion, why is this problem related to the “time constraints” element? Since “time constraints” typically apply to in-game activities, what does it have to do with the post-game discussion? Or was there actually an in-game discussion (which the development team did not mention somehow in the focus group)? Other than being confusing, such an ambiguity also indicates one thing: the design elements may not have actually encompassed every element of the game (understandably, the development team may have forgotten, or chosen not to mention, some actual elements in the focus group for some reason, including because they thought these elements were inessential to their design intentions). If this is the case, the authors should explicitly discuss these “unmentioned elements” to make the “transmission mechanism” of the design intentions clearer.
Regardless of these ambiguities, however, the manuscript’s novelty and contributions remain worthwhile. Thus, we would recommend the manuscript’s acceptance if the authors could resolve the issue.
Citation: https://doi.org/10.5194/egusphere-2024-46-RC1 -
AC1: 'Reply on RC1', Mo Hamza, 25 Mar 2024
The first question that the reviewer poses relates to ambiguity around the game elements. We recognize that outlining the connections in a table instead of elaborating upon them in the text is most likely the cause of this ambiguity. The reason for this choice was to add this layer of complexity, which can help other scholars and game designers to follow and compare the design elements, while avoiding making the main text too heavy and difficult to follow. Therefore, we are thankful for this observation, as it suggests that the elements as they are currently described leave too much room for interpretation for a reader who has never interacted with the game.
In order to make this section more helpful, we propose expanding the descriptions of the elements in the annex to showcase the logical connections, which now remain murky. Addressing the specific examples that the reviewer brings up: The “medium: board” element is intended to transmit relatability with how the board is visually constructed. On the game board, the players recognise familiar concepts and structures from their everyday life, intended to help with forming a connection between the game scenario and the player’s actual life. The “uncontrollable events” element, on the other hand, intends to transmit relatability in the sense that it depicts climate change from a point of view that resembles the experience of the public. As in real life, the causes and effects of climate change cannot only be locally influenced, and the players lack control. Instead of giving the players tools to “change the world”, a more realistic scenario is presented, where disruptions from climate change keep mounting even if the team would put all of their efforts in to stop it from happening.
The second set of questions pinpoints ambiguity emerging from a specific finding, namely that the participants associate the “time constraints” element with lower quality in-game discussions. We recognize how this can lead to confusion, given that discussion was defined by the game developers as relating to the post-game debrief. While it seems that in-game discussion is completely omitted as a design element, and that it appears out of the blue into the discussion section of this paper, the fact that the game was designed to be collaborative - requiring rule-learning and joint decisions - suggests that in-game discussion was taken as a given by the developers.
While the in-game discussion element may be assumed to be included in the game format itself, the observation brought forward by the reviewer is a very interesting one. As the game element is mentioned by several participants as a limiting factor, it begs the question if the development team did not anticipate that the in-game discussion element would have such an impact, and perhaps that the discussion as it is currently structured in the gameplay should be rethought in the next iteration, or by other game developers. This is indeed an emergent quality as suggested by the reviewer, and while the paper currently scratches the surface of this question, it does not assess it with the depth that it deserves, as it is an excellent point of how the game experience deviates from the structure that was developed in the design phase. In order to make this discussion more prominent, and to reduce the ambiguity, we will format the focus group method section to include this discussion on elements that may not have been captured by the method (this particular example included). We will also deepen the argumentation around this finding in the discussion section.
Will submit a new version of the article in which we will execute the above noted changes once we get the comments from the second reviewer in order to include all changes in one complete and final version.
Citation: https://doi.org/10.5194/egusphere-2024-46-AC1
-
AC1: 'Reply on RC1', Mo Hamza, 25 Mar 2024
-
RC2: 'Comment on egusphere-2024-46', Johan Schaar, 05 Apr 2024
General Comments
This is an important and novel contribution to research on the use of games to build awareness and insights on a highly complex issue. It expressly addresses the knowledge-action gap and barriers that stand in the way of acting on insights. It critiques the limitations of narrowly cognitive-focused knowledge-transfer approaches and seeks to understand designer-participant interaction with a critical focus on less studied affective and relational aspects that are often the keys to transformational learning. The approach is particularly important today given the troubling situation that neither mitigation nor adaptation action, at all levels, are keeping pace with the very tangible impacts of the rapidly changing climate. New approaches are needed to understand what prevents implementation of effective policies Games offer a promising tool both to gain a better understanding of impediments and find new ways forward. In this, the paper provides an account and learning from a promising test from which conclusions can be drawn for "guidelines for successful engagement.
The paper provides a very valuable, comprehensive and up-to-date review of the interdisciplinary research field
Specific comments
For the reader less familiar with the world of games, it would be useful if a brief but slightly more detailed description of how the game is provided could be given.
The participating groups are largely self-selected with no attempt to design randomized or representative groups. The consequences of this could be commented on.
The fact that participants represent a number of distinct and different institutions should allow some comparison between them in terms of perceptions, game outcomes and conclusions, both from a focus group and participant perspective.
The 18 game events have taken place in a number of countries and in 4 continents. This indicates that the game is so general in character that it can successfully be introduced in very different contexts. A logical next step to increase its relevance as a tool that can create the foundations for real decision-making would be to adapt it to much more national/local circumstances. For example, it would be of much interest to see the game used in addressing close to real life and concrete trade-offs and tension of which there are many, as indicated in the description of the plan. It would be interesting to see the authors' views of how this kind of application in specific governance settings and bio-physical and social contexts could be envisaged. Can it help decision-making under real-life uncertainty?
Technical corrections
There are few technical issues. The paper is long, probably prohibitive for many potentially interested readers. This could be remedied with a slightly more elaborate abstract.
Line 184: it says that the paper adopts a mixed-method approach. My understanding is that by mixed methods we usually mean the use of both qualitative and quantitative methods. But what is presented are different qualitative methods, not mixed methods.
216. "A careful design of focus groups is key...". But the authors have not designed the focus group but have had to work with those that had actually acted as designers and facilitators.
Citation: https://doi.org/10.5194/egusphere-2024-46-RC2 -
AC2: 'Reply on RC2', Mo Hamza, 07 May 2024
The reviewer highlights that there may be some confusion for readers who are less familiar with games and that there is an inclusivity benefit if we provide more detail when describing the game. We believe this is easily achievable and will continue to keep the length of the manuscript in mind when doing this. Related to this is the reviewer's note under “technical corrections” that a slightly more elaborate abstract would be beneficial because the paper could be prohibitively long for some readers. We agree with this comment and will revisit the abstract so that it better encapsulates the main messages of the paper, which will hopefully make it more accessible.
A second theme in the review is around the participating groups. They note that groups are largely self-selected and therefore there isn't any randomization, nor are there any representative groups included. We would like to highlight that there is a degree of randomisation, given that all game events from which data could be retrieved are included in the sample, and there is no active selection of participants from the part of the researchers, nor the game facilitators. At the same time, we recognise that the groups may have been predisposed for success based on the circumstances that led them to (voluntarily) participate in the first place. This could cause the results of this study to be positively skewed, if compared to a representative control group. Whilst this is a possible explanation for the results, we do see variance in the data, and some participants imply not having much experience with games prior to playing Minions of Disruptions. This gives us confidence that the audience was diverse to some degree. Unfortunately, the kind of data that would have quantitatively mapped out the spectrum of prior experiences and knowledge was not collected, and could not be retrieved anymore at the data analysis phase. We briefly mention this in Section 3.3. Analysis, however, we agree that this could be better elaborated on in the paper, and we will address this as a limitation in the method section as well as the conclusion emphasising that the results are not necessarily generalizable to all groups.
Furthermore, the reviewer brings up that there is a missed opportunity as the paper does not draw comparisons between groups and institutions. We fully agree that this would have made for an interesting analysis, and it was a consideration during the drafting phase. However, given anonymised data, and the limited sample size, we concluded that the results of this comparison would not have been reliable, most notably, for the reasons described above. Given that the researchers did not design the data collection phase, but received third-party data, we did not want to draw conclusions which could generalise groups, if we did not have a sufficiently robust method that would allow us to understand the diverse backgrounds of the participants. Thus, we drew an analytical boundary and focused on the interaction between designers and participants. It is our hope, however, that this paper inspires research methods and approaches in the future that are tailored to study reactions between groups, as we are convinced that this would lead to highly interesting results for the field. We can crystallise this reasoning in the paper in the conclusion tying it to the discussion on future research directions.
Finally, we agree that there could be an opportunity to increase Minions of Disruptions' relevance as a tool for decision-making by adapting it to local circumstances. In fact, this has already happened. In 2022 Day of Adaptation worked to co-design an adapted version of this game to be used with primarily pastoralist communities in rural Kenya. The adapted version of the game was based on real expressed tensions between sustaining livelihoods through climate impacts and decisions made at the hyper-local level that could help communities increase their adaptive capacity in a rapidly changing climate. Using the game as a communication tool and a way to spark difficult conversations showed promise in this specific context, and could likely aid in beginning conversations of community or organisational adaptation in other circumstances, as well. While we feel in-depth discussion of this application is outside the scope of this particular manuscript, we can include a nod to this project as a real-life implication of this work.
We appreciate these comments and technical corrections on the research method (not mixed methods, but different qualitative methods) and focus group design, and will address them in the updated manuscript.
Citation: https://doi.org/10.5194/egusphere-2024-46-AC2
-
AC2: 'Reply on RC2', Mo Hamza, 07 May 2024
-
CC1: 'Comment on egusphere-2024-46', David Crookall, 10 Apr 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-46/egusphere-2024-46-CC1-supplement.pdf
-
AC3: 'Reply on CC1', Mo Hamza, 09 May 2024
Structure:
The reviewer makes several good suggestions to improve the structure, i.e. inverting the order of the title and switching around in the methods section. We agree with these suggestions, and believe this would help with making sure the more important information comes first. Moreover, we see the reviewer’s point that the introduction could benefit from some added detail around the method, as it is now mostly focused on the layout of the paper. We will make sure to review the introduction so that a reader will have a clearer picture of the methods of the paper when they start reading.
The reviewer also makes the suggestion of adding a numbered list of sections at the end of the introduction, and using an abbreviation for “Minions of Disruptions”. We agree that this would make the paper a little bit lighter to read, and are happy to edit accordingly. They also suggest moving some of the contents to the appendix. When reviewing the paper we will make sure to check which features could be left in the appendix, without compromising information that the reader needs in order to follow the flow of the paper.
Language:
The reviewer describes aptly how designers tend to think that the only thing that matters is their design, even if every game experience is dependent on what the participants bring to the table. We completely agree that games are co-constructed experiences, and are happy to emphasise this even more than we do at the moment. Interrelatedly, the reviewer brings up that we contradict ourselves by stating that games are communication vessels transmitting messages. What we mean, and intend, by that is to be transparent about the designers’ role in the process. However, we understand the concern that this makes game design appear a tad too deterministic, and we will make sure to crystallise the thinking here.
We thank the reviewer for bringing up that by using the word dialogue, we might confuse the reader to think about dialogic teaching methods. This is of course not what we are referring to, but the reason for this choice was to emphasise the participant feedback as something that feeds back into the development of the game. We will adjust the language and use the word ‘conversation’, as well as ‘comparison’, where applicable. In the same line of thought, we agree that “received” may not be the most fitting word to describe the participant's experience, and we are happy to adjust this, for instance, to “perceived”. Finally, the reviewer asks what we mean by “highly complex communication”. We agree that perhaps a more fitting term would be simply “complex communication”.
Content:
The reviewer asks what we mean by the fact that the game can be “won”, which stands in contrast to how collaborative games tend to be structured. The answer is two-part. Firstly, we mean that there are no lose scenarios in the game. The purpose of this, according to the game designers, is to create an experience where whichever action for the community leads to positive impact (i.e. a metaphor for community action). The second meaning we attach to winning are the dynamics between playing groups - something that was not designed, but rather what has emerged in the game sessions. The teams are not necessarily aware in the beginning that there are no lose-scenarios. Consequently, competitive elements have emerged between teams playing in the same room. This has led to an excitable atmosphere, and is key in making the game engaging.
The reviewer makes valuable points about debrief. We agree that this is an important part of the game, as well as replicability, and while it is already mentioned in the paper, we are happy to expand on how the debrief was actually organised.
The reviewer asks if the design intentions were developed before the game was finalised or once the game had been used: This is an interesting question, and the answer is two-fold: Firstly, many of the game designers do not facilitate the game, so coming into the focus group session they did not know clearly how the game has been perceived by the audience. Secondly, as we witness a discrepancy between what the game designers intended and parts which were positively perceived by the participants, we are not concerned that the designers’ responses would have been colored with confirmation bias. That being said, we are certain that the design intentions have somewhat evolved over time, as perception is not static. If the designers were asked the same question immediately after the game was created the answers could have varied. For the purpose of this paper, we don’t think that this compromises the message we intend to convey.
The reviewer is also interested to understand the game element better, and see which ones contributed to the primary objectives the most. Unfortunately, and as mentioned in the last paragraph, the participants did not refer to any specific design elements. We believe this is due to the way the standard survey was formulated, and could be improved for the next iteration of this or similar research. We will add the survey questions as an appendix and also add this comment to make sure that if anyone picks up on the method they can make the necessary adjustments to the survey.
Finally, the reviewer asks, how come we claim that there tends to be few games that display collective / collaborative games, while in their experience such games do exist. This observation of a gap comes from review papers, which are referenced in the manuscript. It is good to get the feedback that there might be more climate games with collaborative aspects, and we hope that in the future they will be increasingly represented in academic literature.
We have covered here, what we considered to be the major points presented by the reviewer. The reviewer also included several minor notes on parts which would be good to expand upon, and where we could have been more specific. We will make sure to review all of those points when we edit the manuscript.
Comments related to technical matters:
The reviewer suggests adding the post-game survey to the appendix of the paper to allow the replication of the paper. We agree that this is a good idea.
Based on the reviewer’s comment on not quite understanding the content in table 2 and fig. 2, fig. 3, table 3, and fig. 4 and directly commenting on wanting to gain a deeper insight into the methods, we are happy to deepen the methods section. We will do our best to make even more clear how themes and statements come to be and the way the numbers were produced.
Other comments:
We are thankful for the suggested literature, as we have not come across the reviewer’s work. Moreover, we agree that in the final segment, the paper could benefit from some added literature. As we review the paper, we will see if the reviewer’s or other work could still be incorporated.
We will also consult with Day of Adaptation about what kind of permissions we have to share pictures of participants playing the game, as this would also make the paper a bit more lively and give a sense of an actual event.
We thank the reviewer for the very thorough review as well as encouraging feedback!
Citation: https://doi.org/10.5194/egusphere-2024-46-AC3
-
AC3: 'Reply on CC1', Mo Hamza, 09 May 2024
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-46', Pratama Atmaja, 27 Feb 2024
The manuscript presents a novel contribution to the important yet emerging body of knowledge of games for complex issue communication. Due to their highly interactive and engaging nature, games are increasingly employed to communicate climate crises and other complex issues to the public. One glaring question regarding this communication method is how to ensure that the message is well-received, and this manuscript offers a preliminary yet compelling answer. It begins with providing a concise overview of the current literature on climate crises and games for climate-related communication and education. It then presents a case study of a specific board game for promoting climate actions. It then shows that the game’s effectiveness can be measured by identifying the developer’s design intentions, design elements, and the players’ impressions of the intentions through a focus group and a survey. The manuscript discusses its research method, data, and results in great detail, culminating in design principles and directions for future research, the former useful for scholars and game developers wanting to replicate the game’s success.
At least that is what the manuscript seems to aim for. Unfortunately, its contributions are held back somewhat by ambiguities. At the core of its case study, the manuscript presents three findings: (1) the developer’s design intentions, (2) the players’ impressions of the design intentions, and (3) some design elements that are supposed to transmit the intentions to the players. The first ambiguity revolves around the first and third findings. Looking at the design elements’ descriptions, it is rather difficult to imagine exactly how some elements transmit some of the design intentions. For example, how exactly do “uncontrollable events” and “medium: board” transmit “relatability? The authors should elaborate on this more to make the findings more informative to other scholars and game developers. Additionally, the players’ impressions of the developer’s intentions are also riddled with ambiguity. For example, when some players complained about insufficient time for a discussion, why is this problem related to the “time constraints” element? Since “time constraints” typically apply to in-game activities, what does it have to do with the post-game discussion? Or was there actually an in-game discussion (which the development team did not mention somehow in the focus group)? Other than being confusing, such an ambiguity also indicates one thing: the design elements may not have actually encompassed every element of the game (understandably, the development team may have forgotten, or chosen not to mention, some actual elements in the focus group for some reason, including because they thought these elements were inessential to their design intentions). If this is the case, the authors should explicitly discuss these “unmentioned elements” to make the “transmission mechanism” of the design intentions clearer.
Regardless of these ambiguities, however, the manuscript’s novelty and contributions remain worthwhile. Thus, we would recommend the manuscript’s acceptance if the authors could resolve the issue.
Citation: https://doi.org/10.5194/egusphere-2024-46-RC1 -
AC1: 'Reply on RC1', Mo Hamza, 25 Mar 2024
The first question that the reviewer poses relates to ambiguity around the game elements. We recognize that outlining the connections in a table instead of elaborating upon them in the text is most likely the cause of this ambiguity. The reason for this choice was to add this layer of complexity, which can help other scholars and game designers to follow and compare the design elements, while avoiding making the main text too heavy and difficult to follow. Therefore, we are thankful for this observation, as it suggests that the elements as they are currently described leave too much room for interpretation for a reader who has never interacted with the game.
In order to make this section more helpful, we propose expanding the descriptions of the elements in the annex to showcase the logical connections, which now remain murky. Addressing the specific examples that the reviewer brings up: The “medium: board” element is intended to transmit relatability with how the board is visually constructed. On the game board, the players recognise familiar concepts and structures from their everyday life, intended to help with forming a connection between the game scenario and the player’s actual life. The “uncontrollable events” element, on the other hand, intends to transmit relatability in the sense that it depicts climate change from a point of view that resembles the experience of the public. As in real life, the causes and effects of climate change cannot only be locally influenced, and the players lack control. Instead of giving the players tools to “change the world”, a more realistic scenario is presented, where disruptions from climate change keep mounting even if the team would put all of their efforts in to stop it from happening.
The second set of questions pinpoints ambiguity emerging from a specific finding, namely that the participants associate the “time constraints” element with lower quality in-game discussions. We recognize how this can lead to confusion, given that discussion was defined by the game developers as relating to the post-game debrief. While it seems that in-game discussion is completely omitted as a design element, and that it appears out of the blue into the discussion section of this paper, the fact that the game was designed to be collaborative - requiring rule-learning and joint decisions - suggests that in-game discussion was taken as a given by the developers.
While the in-game discussion element may be assumed to be included in the game format itself, the observation brought forward by the reviewer is a very interesting one. As the game element is mentioned by several participants as a limiting factor, it begs the question if the development team did not anticipate that the in-game discussion element would have such an impact, and perhaps that the discussion as it is currently structured in the gameplay should be rethought in the next iteration, or by other game developers. This is indeed an emergent quality as suggested by the reviewer, and while the paper currently scratches the surface of this question, it does not assess it with the depth that it deserves, as it is an excellent point of how the game experience deviates from the structure that was developed in the design phase. In order to make this discussion more prominent, and to reduce the ambiguity, we will format the focus group method section to include this discussion on elements that may not have been captured by the method (this particular example included). We will also deepen the argumentation around this finding in the discussion section.
Will submit a new version of the article in which we will execute the above noted changes once we get the comments from the second reviewer in order to include all changes in one complete and final version.
Citation: https://doi.org/10.5194/egusphere-2024-46-AC1
-
AC1: 'Reply on RC1', Mo Hamza, 25 Mar 2024
-
RC2: 'Comment on egusphere-2024-46', Johan Schaar, 05 Apr 2024
General Comments
This is an important and novel contribution to research on the use of games to build awareness and insights on a highly complex issue. It expressly addresses the knowledge-action gap and barriers that stand in the way of acting on insights. It critiques the limitations of narrowly cognitive-focused knowledge-transfer approaches and seeks to understand designer-participant interaction with a critical focus on less studied affective and relational aspects that are often the keys to transformational learning. The approach is particularly important today given the troubling situation that neither mitigation nor adaptation action, at all levels, are keeping pace with the very tangible impacts of the rapidly changing climate. New approaches are needed to understand what prevents implementation of effective policies Games offer a promising tool both to gain a better understanding of impediments and find new ways forward. In this, the paper provides an account and learning from a promising test from which conclusions can be drawn for "guidelines for successful engagement.
The paper provides a very valuable, comprehensive and up-to-date review of the interdisciplinary research field
Specific comments
For the reader less familiar with the world of games, it would be useful if a brief but slightly more detailed description of how the game is provided could be given.
The participating groups are largely self-selected with no attempt to design randomized or representative groups. The consequences of this could be commented on.
The fact that participants represent a number of distinct and different institutions should allow some comparison between them in terms of perceptions, game outcomes and conclusions, both from a focus group and participant perspective.
The 18 game events have taken place in a number of countries and in 4 continents. This indicates that the game is so general in character that it can successfully be introduced in very different contexts. A logical next step to increase its relevance as a tool that can create the foundations for real decision-making would be to adapt it to much more national/local circumstances. For example, it would be of much interest to see the game used in addressing close to real life and concrete trade-offs and tension of which there are many, as indicated in the description of the plan. It would be interesting to see the authors' views of how this kind of application in specific governance settings and bio-physical and social contexts could be envisaged. Can it help decision-making under real-life uncertainty?
Technical corrections
There are few technical issues. The paper is long, probably prohibitive for many potentially interested readers. This could be remedied with a slightly more elaborate abstract.
Line 184: it says that the paper adopts a mixed-method approach. My understanding is that by mixed methods we usually mean the use of both qualitative and quantitative methods. But what is presented are different qualitative methods, not mixed methods.
216. "A careful design of focus groups is key...". But the authors have not designed the focus group but have had to work with those that had actually acted as designers and facilitators.
Citation: https://doi.org/10.5194/egusphere-2024-46-RC2 -
AC2: 'Reply on RC2', Mo Hamza, 07 May 2024
The reviewer highlights that there may be some confusion for readers who are less familiar with games and that there is an inclusivity benefit if we provide more detail when describing the game. We believe this is easily achievable and will continue to keep the length of the manuscript in mind when doing this. Related to this is the reviewer's note under “technical corrections” that a slightly more elaborate abstract would be beneficial because the paper could be prohibitively long for some readers. We agree with this comment and will revisit the abstract so that it better encapsulates the main messages of the paper, which will hopefully make it more accessible.
A second theme in the review is around the participating groups. They note that groups are largely self-selected and therefore there isn't any randomization, nor are there any representative groups included. We would like to highlight that there is a degree of randomisation, given that all game events from which data could be retrieved are included in the sample, and there is no active selection of participants from the part of the researchers, nor the game facilitators. At the same time, we recognise that the groups may have been predisposed for success based on the circumstances that led them to (voluntarily) participate in the first place. This could cause the results of this study to be positively skewed, if compared to a representative control group. Whilst this is a possible explanation for the results, we do see variance in the data, and some participants imply not having much experience with games prior to playing Minions of Disruptions. This gives us confidence that the audience was diverse to some degree. Unfortunately, the kind of data that would have quantitatively mapped out the spectrum of prior experiences and knowledge was not collected, and could not be retrieved anymore at the data analysis phase. We briefly mention this in Section 3.3. Analysis, however, we agree that this could be better elaborated on in the paper, and we will address this as a limitation in the method section as well as the conclusion emphasising that the results are not necessarily generalizable to all groups.
Furthermore, the reviewer brings up that there is a missed opportunity as the paper does not draw comparisons between groups and institutions. We fully agree that this would have made for an interesting analysis, and it was a consideration during the drafting phase. However, given anonymised data, and the limited sample size, we concluded that the results of this comparison would not have been reliable, most notably, for the reasons described above. Given that the researchers did not design the data collection phase, but received third-party data, we did not want to draw conclusions which could generalise groups, if we did not have a sufficiently robust method that would allow us to understand the diverse backgrounds of the participants. Thus, we drew an analytical boundary and focused on the interaction between designers and participants. It is our hope, however, that this paper inspires research methods and approaches in the future that are tailored to study reactions between groups, as we are convinced that this would lead to highly interesting results for the field. We can crystallise this reasoning in the paper in the conclusion tying it to the discussion on future research directions.
Finally, we agree that there could be an opportunity to increase Minions of Disruptions' relevance as a tool for decision-making by adapting it to local circumstances. In fact, this has already happened. In 2022 Day of Adaptation worked to co-design an adapted version of this game to be used with primarily pastoralist communities in rural Kenya. The adapted version of the game was based on real expressed tensions between sustaining livelihoods through climate impacts and decisions made at the hyper-local level that could help communities increase their adaptive capacity in a rapidly changing climate. Using the game as a communication tool and a way to spark difficult conversations showed promise in this specific context, and could likely aid in beginning conversations of community or organisational adaptation in other circumstances, as well. While we feel in-depth discussion of this application is outside the scope of this particular manuscript, we can include a nod to this project as a real-life implication of this work.
We appreciate these comments and technical corrections on the research method (not mixed methods, but different qualitative methods) and focus group design, and will address them in the updated manuscript.
Citation: https://doi.org/10.5194/egusphere-2024-46-AC2
-
AC2: 'Reply on RC2', Mo Hamza, 07 May 2024
-
CC1: 'Comment on egusphere-2024-46', David Crookall, 10 Apr 2024
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2024/egusphere-2024-46/egusphere-2024-46-CC1-supplement.pdf
-
AC3: 'Reply on CC1', Mo Hamza, 09 May 2024
Structure:
The reviewer makes several good suggestions to improve the structure, i.e. inverting the order of the title and switching around in the methods section. We agree with these suggestions, and believe this would help with making sure the more important information comes first. Moreover, we see the reviewer’s point that the introduction could benefit from some added detail around the method, as it is now mostly focused on the layout of the paper. We will make sure to review the introduction so that a reader will have a clearer picture of the methods of the paper when they start reading.
The reviewer also makes the suggestion of adding a numbered list of sections at the end of the introduction, and using an abbreviation for “Minions of Disruptions”. We agree that this would make the paper a little bit lighter to read, and are happy to edit accordingly. They also suggest moving some of the contents to the appendix. When reviewing the paper we will make sure to check which features could be left in the appendix, without compromising information that the reader needs in order to follow the flow of the paper.
Language:
The reviewer describes aptly how designers tend to think that the only thing that matters is their design, even if every game experience is dependent on what the participants bring to the table. We completely agree that games are co-constructed experiences, and are happy to emphasise this even more than we do at the moment. Interrelatedly, the reviewer brings up that we contradict ourselves by stating that games are communication vessels transmitting messages. What we mean, and intend, by that is to be transparent about the designers’ role in the process. However, we understand the concern that this makes game design appear a tad too deterministic, and we will make sure to crystallise the thinking here.
We thank the reviewer for bringing up that by using the word dialogue, we might confuse the reader to think about dialogic teaching methods. This is of course not what we are referring to, but the reason for this choice was to emphasise the participant feedback as something that feeds back into the development of the game. We will adjust the language and use the word ‘conversation’, as well as ‘comparison’, where applicable. In the same line of thought, we agree that “received” may not be the most fitting word to describe the participant's experience, and we are happy to adjust this, for instance, to “perceived”. Finally, the reviewer asks what we mean by “highly complex communication”. We agree that perhaps a more fitting term would be simply “complex communication”.
Content:
The reviewer asks what we mean by the fact that the game can be “won”, which stands in contrast to how collaborative games tend to be structured. The answer is two-part. Firstly, we mean that there are no lose scenarios in the game. The purpose of this, according to the game designers, is to create an experience where whichever action for the community leads to positive impact (i.e. a metaphor for community action). The second meaning we attach to winning are the dynamics between playing groups - something that was not designed, but rather what has emerged in the game sessions. The teams are not necessarily aware in the beginning that there are no lose-scenarios. Consequently, competitive elements have emerged between teams playing in the same room. This has led to an excitable atmosphere, and is key in making the game engaging.
The reviewer makes valuable points about debrief. We agree that this is an important part of the game, as well as replicability, and while it is already mentioned in the paper, we are happy to expand on how the debrief was actually organised.
The reviewer asks if the design intentions were developed before the game was finalised or once the game had been used: This is an interesting question, and the answer is two-fold: Firstly, many of the game designers do not facilitate the game, so coming into the focus group session they did not know clearly how the game has been perceived by the audience. Secondly, as we witness a discrepancy between what the game designers intended and parts which were positively perceived by the participants, we are not concerned that the designers’ responses would have been colored with confirmation bias. That being said, we are certain that the design intentions have somewhat evolved over time, as perception is not static. If the designers were asked the same question immediately after the game was created the answers could have varied. For the purpose of this paper, we don’t think that this compromises the message we intend to convey.
The reviewer is also interested to understand the game element better, and see which ones contributed to the primary objectives the most. Unfortunately, and as mentioned in the last paragraph, the participants did not refer to any specific design elements. We believe this is due to the way the standard survey was formulated, and could be improved for the next iteration of this or similar research. We will add the survey questions as an appendix and also add this comment to make sure that if anyone picks up on the method they can make the necessary adjustments to the survey.
Finally, the reviewer asks, how come we claim that there tends to be few games that display collective / collaborative games, while in their experience such games do exist. This observation of a gap comes from review papers, which are referenced in the manuscript. It is good to get the feedback that there might be more climate games with collaborative aspects, and we hope that in the future they will be increasingly represented in academic literature.
We have covered here, what we considered to be the major points presented by the reviewer. The reviewer also included several minor notes on parts which would be good to expand upon, and where we could have been more specific. We will make sure to review all of those points when we edit the manuscript.
Comments related to technical matters:
The reviewer suggests adding the post-game survey to the appendix of the paper to allow the replication of the paper. We agree that this is a good idea.
Based on the reviewer’s comment on not quite understanding the content in table 2 and fig. 2, fig. 3, table 3, and fig. 4 and directly commenting on wanting to gain a deeper insight into the methods, we are happy to deepen the methods section. We will do our best to make even more clear how themes and statements come to be and the way the numbers were produced.
Other comments:
We are thankful for the suggested literature, as we have not come across the reviewer’s work. Moreover, we agree that in the final segment, the paper could benefit from some added literature. As we review the paper, we will see if the reviewer’s or other work could still be incorporated.
We will also consult with Day of Adaptation about what kind of permissions we have to share pictures of participants playing the game, as this would also make the paper a bit more lively and give a sense of an actual event.
We thank the reviewer for the very thorough review as well as encouraging feedback!
Citation: https://doi.org/10.5194/egusphere-2024-46-AC3
-
AC3: 'Reply on CC1', Mo Hamza, 09 May 2024
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 411 | 118 | 52 | 581 | 12 | 11 |
- HTML: 411
- PDF: 118
- XML: 52
- Total: 581
- BibTeX: 12
- EndNote: 11
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Minja Sillanpää
Ana Capri Mauro
Minttu Hänninen
Sam Illingworth
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(982 KB) - Metadata XML