the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 27 Nov 2023
Guiding community discussions on human-water-related challenges by serious gaming in the upper Ewaso Ng’iro river basin, Kenya
Abstract. Water-related conflicts in river catchments occur due to both internal and external pressures that affect catchment water availability. Lack of shared understanding by catchment stakeholders increase the complexity of human-water issues at the river catchment scale. Among a range of participatory approaches, the development and use of serious games gained prominence as a tool to stimulate discussion and reflection among stakeholders about sustainable resource use and collective action. This study designed and implemented the ENGAGE game (Exploring New Gaming Approach to Guide and Enlighten), that mimics the dynamics observed during the dry season in the upper Ewaso Ng’iro catchment, North West of Mount Kenya. The purpose of this study was to explore the potential role of serious gaming in subsequent steps of strengthening stakeholder engagement (agenda setting, shared understanding, commitment to collective action, and means of implementation) toward addressing complex human-water-related challenges at the catchment scale. We assessed the type of decisions made during gameplay, the communication dynamics, active participation, and the implication of decisions made on water availability. The results of three game sessions show that the ENGAGE game raised awareness and provided a recognizable hydro-logic background to conflicts while guiding community discussions toward implementable decisions. The results revealed increasing active participation, knowledge gain, and use of plural pronouns, and decreasing individual interests and conflicts among game participants. This study presents important implications for creating a collective basis for water management and can inform human-water policies and modification of the process behind water allocation rules in a river catchment.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(2894 KB)
-
Supplement
(1259 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2894 KB) - Metadata XML
-
Supplement
(1259 KB) - BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
CC1: 'Comment on egusphere-2023-2459', Wim Douven, 14 Dec 2023
Dear Charles and team, I enjoyed reading your manuscript. I find the topic very interesting, and it is good to have another study on serious games. Below some comments, questions, that might be helpful in finalising the manuscript.
In the section on IWRM (line 66 and further), you mention that workshops do not promote different experts to engage with wicked problems. But doesn’t that depend on how you design such workshops, like how you design serious games ..?
I note that the lit. review on games is overall very positive on its effects on finding joint solutions etc.., are there no limitations and challenges reported in literature ? You mention one in your discussion in line 520. Did you come across other limitations, challenges in the sessions?
In game conceptualisation (Section 2.2), it might help the reader to give a brief summary about sessions, rounds, and phases, and how they relate to the catchment / sub-catchments.
Data collection (Section 2.4) is there after the step 'modeling the game solution space' not also a step where you collect data about discussion and feed-back, which might result in adjustments of positions and solutions .? As it is an interactive process, it is collecting data and providing data, right? (so feed-back, and giving space for learning)?
An important discussion point you touch upon is how the game results translate to real life situation (line 560 and further). I wondered was this discussed in the sessions, at the end? How the game was valued, also for real-life? That could already give some initial insights?
Citation: https://doi.org/10.5194/egusphere-2023-2459-CC1 -
AC1: 'Reply on CC1', Charles Wamucii, 22 Dec 2023
CC1: 'Comment on egusphere-2023-2459', Wim Douven, 14 Dec 2023
Dear Charles and team, I enjoyed reading your manuscript. I find the topic very interesting, and it is good to have another study on serious games. Below some comments, questions, that might be helpful in finalising the manuscript. I am happy to hear you enjoyed reading the manuscript. Thank you for the positive remarks on the manuscript and constructive suggestions and comments.
Comment 1: In the section on IWRM (line 66 and further), you mention that workshops do not promote different experts to engage with wicked problems. But doesn’t that depend on how you design such workshops, like how you design serious games ..? Thank you for the question.
Response: Yes, we agree that that there are many different participatory methods developed especially to promote the engagement of 'local experts'. One of the most familiar is focussed groups, but many others are around including Dramas. Designing serious game sessions may differ from designing workshop sessions. Firstly, serious games are co-designed where both the ‘scientists’ and ‘local experts’ are involved in the development of serious games (way before actual game sessions). In the workshop sessions, the scientists’ are the most active persons, e.g. when making PowerPoint presentations, explaining an idea, reporting case studies, or simulating a model and its possible results. The participation of ‘local experts’ is relatively ‘passive’ in the workshop sessions e.g. sitting and listening to presentations made by scientists. Some workshops may divide participants into small groups for discussions, experimenting, and demonstrating workshop issues. But in most cases, the group discussions tend to have active and less active persons - in Ln 547 of the manuscript, we have argued this could be due to power differentials among participants… Workshops often are not the right way to deal with wicked problems.
There is a difference in the game sessions. Serious games provide a 'buffer' to mitigate powerplay and hierarchy, make it more easy for participants to express more freely, since it is 'just' a game. Serious games game provide a boundary object commonly not found in workshops. The ‘scientists’ are less active, and tend to play the passive role of facilitating the process (especially at the start of the game rounds), while participants are the most active persons (in small groups) right from the start to the end of the sessions. The facilitation role tends to decrease as the game progresses, and at some point ‘scientists’ become the ‘observers’ of what is happening in the different small groups of gameplay. Scientists tend to learn more this way, as opposed to when they are the main presenters in conventional workshops. The game participants are vigorous as they interact directly with ‘virtual systems’ such as Board games and also actively engage one another (e.g. disagreeing, setting rules, seeking alternative strategies, etc), the scientists do not control the actual gameplay. In Ln 542 of the manuscript, we have argued that ‘…..since each participant in the game has a role to play, they have a stake during the game. They tend to be active in pursuing their stake while focusing on their roles…’
We agree that if we design workshops similarly to game processes (including having game-like engagements during small group discussions), then designs of workshops and game sessions could both promote different experts to engage in a meaningful way. Still, in most conventional workshops the ‘scientists’ are the main ‘active persons’ while ‘local experts’ are the ‘passive’ participants. By design game sessions reverse these roles.
Comment 2: I note that the lit. review on games is overall very positive on its effects on finding joint solutions etc.., are there no limitations and challenges reported in literature ? You mention one in your discussion in line 520. Did you come across other limitations, challenges in the sessions? Thank you for the questions.
Response: This study focussed on the literature that has attempted to report on the ‘effectiveness of gaming approach’. Hence the purpose of this study was to ‘investigate these opinions/claims’ by assessing the potential role of serious gaming in strengthening stakeholder engagement. This study contributes to the debate on the ‘effectiveness of serious gaming’ including how some results contradict past studies see Ln 466-476.
We have highlighted a few limitations in this study (given the focus of the manuscript) in Ln 500 – 521. Definitely, we can probe more about the limitations and challenges of serious games (in other general/specific game settings) and incorporate them in the discussion section (limitations sub-section of this manuscript). Generally, among the limitations/challenges, the most important one is: you cannot play with people if they are in the mood to fight. So if there already is a conflict, that needs to be dealt with first. Another complication can be that some people have their reservations to engage in a game session if particular other individuals also participate - for instance, bosses with employees, in a hierarchical society.
Comment 3: In game conceptualisation (Section 2.2), it might help the reader to give a brief summary about sessions, rounds, and phases, and how they relate to the catchment / sub-catchments. Thank you for the suggestion.
Response: During this stage of game conceptualization, there was no playing the games per se, but this is the ‘setting stage’, which involved bringing all possible ideas to the table, to help in crafting a serious game that fits the context of the case study area. The ideas were gathered from past studies, through the ARDI approach, and conducting community discussions with communities in three sub-catchments. Game testing sessions are described in sub-section 2.3, where feedback was key to help in refining the final game.
Comment 4: Data collection (Section 2.4) is there after the step 'modeling the game solution space' not also a step where you collect data about discussion and feed-back, which might result in adjustments of positions and solutions .? As it is an interactive process, it is collecting data and providing data, right? (so feed-back, and giving space for learning)? Thank you for the questions.
Response: The data collection description in sub-section 2.4 focussed on data collection based on the final ENGAGE game_v1 described in Box 1 and Supplement 1. To ensure comparability of data across different game sessions, no further adjustment was made and v1 of the game was maintained (as developed) in the three sub-catchments. Yes, there is a feedback session at the end of playing the games, where participants are allowed to give their feedback and key lessons on the game sessions. The qualitative feedback was useful in crafting some of the reflections included in the discussion section e.g. in Ln 523 … the game environment allows for real-time reflection through the creation of a fictional setting and a common pool for the stakeholders to explore decisions and impacts simultaneously’……in Ln 542 …….’each participant in the game has a role to play, they have a stake during the game. They tend to be active in pursuing their stake while focusing on their roles’…….
This manuscript focusses on the ‘decisions made and qualitative aspects of gameplay’ i.e. sentiments raised during the actual gameplay. It is paramount to mention that “another manuscript (also reporting on 5 additional game sessions” focusses on the experiential learning of ENGAGE game_V1, and how stakeholders' perspectives change before and after game sessions, including the post-game individual interviews with different game participants is about to be submitted.
Comment 5: An important discussion point you touch upon is how the game results translate to real life situation (line 560 and further). I wondered was this discussed in the sessions, at the end? How the game was valued, also for real-life? That could already give some initial insights? Thank you for the questions.
Response: Yes, a conversation on translating game experience to real-life situations was a key discussion point at the end of the game sessions. There was high positivity on the usefulness of the gaming approach in promoting sustainable behaviors, and all game participants indicated they would adopt a few lessons from the game. The following are some of the direct quotes from some of the game participants:
- “I need to reduce dependency on the river, I need to construct a small dam”
- “I need to manage the land use better, considering the wet and dry seasons. I also need to construct a small dam”
- “I need to stop clearing the bushes and forest for agricultural land expansion, I better manage what I already have”
However, this manuscript recommends further follow-ups to assess the impact of serious games on strengthening stakeholders' engagement and maintaining sustainable behaviors in real life.
Follow-up data collection was done in another study (“The Manuscript on how stakeholders' perspectives change before and after serious games”), whereby game participants were traced 5-7 months after the game sessions, and most of these aspects (game experience – real life situation) have been explored.
-
AC1: 'Reply on CC1', Charles Wamucii, 22 Dec 2023
-
RC1: 'Comment on egusphere-2023-2459', Jean-Philippe Venot, 01 Mar 2024
This is a paper that investigates the use of a serious game to discuss issues of water management at watershed level in Kenya. I enjoyed reading the paper; a lot of work has gone into the design, implementation and analyze of the research. I think this type of paper reflecting on the use and scope of such participatory approaches for improving water governance is important.
That being said, I feel the paper shares what I see as a bias in the existing literature on games for sustainable natural resources management, that is, a rather positive outlook on the changes/transformations these games may trigger and a focus on their promises and potentials – that is only partly supported by what is actually described in the papers (that is what happens during “game sessions” as opposed to “in the real world”). In that specific case, I think the authors could (1) better clarify how the process they steared was fundamentally different (or not) from other more classic participatory approaches notably in relation to co-production of the tool (as opposed to a contrasting active game with passive attendance to workshop). They could also (2) engage a bit more with the risks/limitations of games that have been discussed in the literature and provide a bit more information on the (3) methodology (including the link between the game and the computer model) and (4) the mechanics of the game in the core of the paper (fez readers are likely to read the supplementary material) and (5) finally not only discuss the fact that the game seem to have enhanced communication among participants and their promises in general, but also issues related to the politics of the game themselves, that is, the extent to which the representation of the system was accepted/validated/discussed by participants and the scope for the game to change the broader rules of the game and conditions of operation of the WRUAs.
I come back to these points one by one below.
Major Comment 1. Line 66 and following: The authors attribute some of the shortcomings of IWRM to the fact that IWRM projects and policies make use of “classic” participatory approaches that would not really allow different actors to engage with the “wicked problem” at hand. In other words, participation in IWRM is not real participation. This is a well-known critique of IWRM, which I share in part, but I think there is a danger of building a strawman for the wrong reasons here, just to make the point that games do not share these shortcomings. I agree when the authors answer the previous community reviewer that games allow a different, more active, mode of engagement than classic workshop based that use power point presentation. But there are many ways to conduct workshop that can provide meaningful engagement of participants: participatory mapping, experimentation with art-based visual, etc. are some of these options.
I think the issue is less about the modalities of facilitation of workshops (and the tools used) than how multiplicity of knowledge is handled and how/who decides on the terms of the debate. Notably, who ends up deciding what the ‘wicked problem’ is. Games can actually reify the ‘wicked problem’ on the basis of an outsider/expert view; if so I would argue that the nature of engagement of participants remains as limited as in other approach or at least is constrained in major ways.What bothers me is that the phrasing of this paragraph (and many others) seems to indicate that the authors hold the view that there is ONE “wicked problem” (likely a lack of coordination between downstream and upstream users). It makes it seem as if this problem existed almost independently of the people who participate, being identified à priori by a subset of actors (though there is scope for refining the problem at hand through the use of the game itself). If that is the case, then, the game developed is likely to have very similar shortcomings than “classic” IWRM based approach.
I think the critique of IWRM approaches would be more powerful and convincing if it were to be done at the level of how are the terms of the debate/participation set and by whom (as opposed to looking at the specifics of interactions and their presumed passivity), and then to show, how this particular study did something different, hence created the room for another type of engagement by the people who participated in the game session. The point here is to pay a bit more attention to the politics that framed the entire research process and operated silently in the background as opposed to take the dynamism of the game sessions as a sign of pro-active engagement and changes to come in the watershed. The authors may want to look at https://hess.copernicus.org/preprints/hess-2023-164/ currently under review in the same journal. This paper reflects on some of those issues around the practices of hydrological modeling not games, but I think there are many similariries.
I do not doubt the authors have done something quite different from “classic passive workshops”; my qualm is with the way authors describe the differences. Instead of making clear what the differences were, the way it is written points to fundamental similarities in the assumptions underlying the approach (which may not have been there). I feel clarifying this point through some reformulation is important and would strengthen the argument of the paper.
Major Comment 2. Line 93 and following. The way to depicts serious games and what they can help achieve is overwhelmingly positive. I think this reflects rather well the existing literature on serious games, most of which is actually written by people involved in their design and implementation – which may make them a bit less likely to adopt a critical stance towards these tools. Yet, I think the authors could engage with some of the limits of games that have been discussed in the literature, notably by members of the ComMod collective they are likely to know. There is some work in relation to conflicts and existing relations of domination (Barnaud & Van Paassen, 2013; Bécu et al., 2008; Mathevet et al., 2014), on the roles and postures of facilitators and modelers (Barreteau et al., 2003; Jonsson et al., 2007), or the risk of manipulation (Barnaud et al., 2008; Halbe et al., 2018). A recent overview recognizes ‘‘potential biases” (Barreteau et al., 2021). See also (Jonsson et al., 2007) on whether games are “truly participatory”. Also, the fact that games simplify complex problems (line 102) is presented as a positive thing (I suspect in relation to legibility of the said problem). This may be the case, but it also raises a lot of questions: who decides what is represented and what is not, what are the implications of those choices… In other words, what are the politics of the game itself? These questions ought to be at least mentioned I think.
Major Comment 3. I feel there is a need to clarify the methodology. Notably, in the first paragraph it is not clear to me what the “solution space” of the game (what are ‘responses’ to rules?). It is also not clear what the authors mean by “overall performance of the game results” (how is performance assessed, is it in relation to water availability/sharing? In relation to how active participation was…)? In relation to that, section 2.4.3 discusses a modeling of the ‘game solution space’ and mentions a 1000 runs but what are those runs? I assume the authors used both a physical game and a computer version of it and that they used the later a 1000 times generating random actions; this to situate the results of the specific game sessions in the “realm of possible”. But I can only assume this: the relations between game and model need to be made explicit and the authors need to explain, why the use of the model was useful, what did it allow to do?
Major Comment 4. A lot of key information on the mechanics of the game (including how the connections between actors and water flows are conceptualized) are included in the supplementary material. I think only few readers are likely to read the supplementary material and it might be a good idea if the authors extended the presentation of the game in the core of the paper slightly. I’m also not sure the game should be presented in a “box”. Normal section numbering would do. In terms of presenting the game, I think it would be interesting to have a sub-section describing the board and the players, another one describing briefly the different action/decision each actor can make and how this impact water flows (at a conceptual level; not entering in the details of the calibration), and another one on the “typical” process followed during each session – this is already in large part in the box. I think the graph on the system dynamic modeling could support this presentation of the game, as opposed to be presented in relation to the “solution space” of the game. The supplementary file can then focus on the calibration aspect. Presenting the game as such would also allow having part of the discussion engaging with the key stage of game validation that is not really described at the moment (what hypothesis the authors made were validated, which one were not…? for instance deciding that water dries up from downstream when people abstract water from upstream is a strong choice. I would be curious to know if that triggered some discussion also because, materially, it involves taking a marble from one area of the board to put it in the other one, which is not really how water abstraction works in the real world.)
Major comment 5. The discussion focuses on what has happened during the game session and it is important. The game sessions are likely to have improved communication and cross learning among participants but to which extent did it actually provide ways to solve the structural issues faced by WRUA that are identified in line 188 (weak enforcement of policies/laws, water abstraction regulations, water metering requirements, protection of riparian corridors/forested areas, etc.)? Such structural issues are likely to relate to the institutional strength of the WUAs, how legitimate it is vis-à-vis other actors, the (human) and financial means it has… Even if it is not the objective of this paper to assess the scope of the game to change things in the real world, I think it is important for the authors to engage in a discussion around the promises and potential of games in relation to these structural hurdles.
Most of the literature on games tend to focus on the design process and on what is happening during game sessions, and that’s why most of the literature is positive: there is a lot of interesting things happening, some of which do have transformative potential. But in doing so, the game and the overall research process is also “removed” from the overall political-economic context in which it is implemented while this context is pivotal for the promises of the game to actually materialize or not. While describing the overall political-economy that influences water management in the case study area is beyond the scope of this paper, I think it is important the authors reflect a bit on it and how this might influence the future of the game outcomes – to avoid falling in the trap of highlighting promises that may never materialize.
I also have some specific comments:
Line 21 (Abstract): What do the authors mean by “shared understanding”, and understanding of what exactly? Could it be clarified?
Line 27 (abstract): This sentence seems to point to the fact that the objective of the study was to assess the potential of the game to strengthen stakeholder engagement “next” (that is, after the game sessions were held). I do not think the paper demonstrates this. It demonstrates rather convincingly that there was active engagement during the game session but does not engage with whether or not this translated in strengthened engagement outside of the game session. I understand from the exchange with the community member who commented on the paper that this is not the point of this particular paper but then, maybe this sentence needs to be rephrased to indicate that what is being described in the dynamics of engagement during gaming session. Similarly, some clarity may be needed on line 125.
Line 47. The term “wicked problems” was first coined by Horst Rittel and Melvin Webber in a 1973 paper, which would be good to mention.
Line 54 to 56: It is not clear how the 5 interacting phases of public debate had a bearing on the research conducted. Why do the authors refer to this work, and how useful is it to understand the process of designing and implementing game sessions? This is clarified around line 95. Consider slight restructuring on when this is mentioned so as to avoid repetitions?
Line 58 and following: I would not call IWRM an “approach”. In my view it consists rather in a policy and discursive model than an approach per se.
Line 63: can the authors briefly say what successes were actually achieved? In what terms was implementation successful and in which contexts?
Line 107: improved efficiency of what?
Line 101: what does “this” relate to?
Line 112 and following: I am not sure what those sentences are meant to bring to the paper. They look as quite general statements in relation to games (what others have said on games), but it is not clear how much of this is actually being used in this particular study. Maybe they do not belong here but should come earlier in the paragraph? I have a similar remark on communication: are the two sentences on that specific topic meant to discuss game “in general” or has this topic of communication during game played been a specific entry point of analysis of what has happened in the game session organized in this study?
Line 132: what do the authors mean by “sentiment” and how did they assess it. What do they mean by “active” participation and how was it assessed.
Line 145 and following: why does the discussion focuses on “substractive dynamics” only and not on “constructive dynamics”?
Line 160 and following: how many WRUA are there in the sub basin/case study area? One for each sub-river?
Line 177and following: This points to my first major comment. How was this suite of problems identified and by whom? And more specifically, have the people who participated to the game sessions been involved in the definition of these problems. This needs to be clarified as it is largely from this that one can assess to which extent the process followed is fundamentally different from classic “participatory activities” implemented under IWRM processes.References:
Barnaud, C., Trébuil, G., Promburom, P., & Bousquet, F. (2008). La modélisation d’accompagnement pour une gestion concertée des ressources renouvelables en
Thaïlande. Economie Rurale, 303–304–305, 39–59.Barnaud, C., & Van Paassen, A. (2013). Equity, power games, and legitimacy: Dilemmas of participatory natural resource management. Ecology and Society,
18(2), 21. https://doi.org/10.5751/ES-05459-180221.Barreteau, O., Antona, M., d’Aquino, P., Aubert, S., Boissau, S., Bousquet, F., Daré, W., Etienne, M., Le Page, C., Mathevet, R., Trébuil, G., & Weber, J. (2003). Our
companion modelling approach. Journal of Artificial Societies and Social Simulation, 6(1).Barreteau, O., Abrami, G., Bonte, B., Bousquet, F., & Mathevet, R. (2021). Serious Games. In R. Biggs, A. de Vos, R. Preiser, H. Clements, K. Maciejewski, & M.
Schlüter (Eds.), The Routledge handbook of research methods for social- ecological systems (pp. 176–188). London & New York: Routledge.Bécu, N., Neef, A., Schreinemachers, P., & Sangkapitux, C. (2008). Participatory computer simulation to support collective decision-making: Potential and
limits of stakeholder involvement. Land Use Policy, 25(4), 498–509. https://doi. org/10.1016/j.landusepol.2007.11.002.Halbe, J., Pahl-Wostl, C., & Adamowski, J. (2018). A methodological framework to support the initiation, design and institutionalization of participatory modeling
processes in water resources management. Journal of Hydrology, 556, 701–716. https://doi.org/10.1016/j.jhydrol.2017.09.024.Jonsson, A., Andersson, L., Alkan-Olsson, J., & Arheimer, B. (2007). How participatory can participatory modeling be? Degrees of influence of stakeholder and expert
perspectives in six dimensions of participatory modeling. Water Science and Technology, 56(1), 207–214. https://doi.org/10.2166/wst.2007.453Mathevet, R., Antona, M., Barnaud, C., Fourage, C., Trébuil, G., & Aubert, S. (2014). Contexts and dependencies in the commod processes. In M. Etienne (Ed.),
Companion Modelling: A participatory approach to support sustainable development (pp. 103–125). Dordrecht: Springer.Venot, JP.; Jensen CB.; Delay E. et Daré, W. (2022). Mosaic glimpses: Serious games, generous constraints, and sustainable futures in Kandal, Cambodia. World Development 151 (2022):105779.
Citation: https://doi.org/10.5194/egusphere-2023-2459-RC1 -
AC2: 'Acknowledgement of referee comments:', Charles Wamucii, 05 Mar 2024
I am happy to hear you enjoyed reading the manuscript. Thank you for the remarks on the manuscript and constructive suggestions and comments. I acknowledge receipt of the following issues as the major comments:
- Clarifying how the process was fundamentally different (or not) from other more classic participatory approaches notably in relation to co-production of the tool (as opposed to a contrasting active game with passive attendance to workshop).
- Expand the discussion on the risks/limitations of games that have been discussed in the literature.
- Provide more information on the methodology (including the link between the game and the computer model)
- Expand the game mechanics of the game in the core of the paper (most details are captured in the supplementary material)
- Expand discussion beyond enhanced communication during gameplay to also include game politics.
I will come back with detailed responses to each of these comments and suggestions as soon as possible. I am currently preparing for my PhD defence, which will be held next week on the 13th of March 2024, at 11:00 AM (Amsterdam time) at Wageningen University. This will also be broadcast live here: (https://wur.yuja.com/). Perhaps some of these issues may come up during my PhD defence.
Regards,
Citation: https://doi.org/10.5194/egusphere-2023-2459-AC2 -
AC3: 'Reply on RC1', Charles Wamucii, 25 Mar 2024
Dear referee,
Thank you for the acknowledgment of the relevance of our research exploring the potential of participatory approaches such as serious games for improving water resources
management and governance. In the attachment below, find our responses and how we plan to address your major five comments and specific comments.Regards,
Charles, on behalf of Co-authors
-
AC2: 'Acknowledgement of referee comments:', Charles Wamucii, 05 Mar 2024
-
EC1: 'Comment on egusphere-2023-2459', Pieter van der Zaag, 07 Mar 2024
I herewith would like to note that the Community Comment by Wim Douven (CC1) should be considered to be a Referee Comment, as I had invited and nominated Wim Douven as a referee for this article. For some reason his comment was labelled as a CC and not as an RC.
Citation: https://doi.org/10.5194/egusphere-2023-2459-EC1
Interactive discussion
Status: closed
-
CC1: 'Comment on egusphere-2023-2459', Wim Douven, 14 Dec 2023
Dear Charles and team, I enjoyed reading your manuscript. I find the topic very interesting, and it is good to have another study on serious games. Below some comments, questions, that might be helpful in finalising the manuscript.
In the section on IWRM (line 66 and further), you mention that workshops do not promote different experts to engage with wicked problems. But doesn’t that depend on how you design such workshops, like how you design serious games ..?
I note that the lit. review on games is overall very positive on its effects on finding joint solutions etc.., are there no limitations and challenges reported in literature ? You mention one in your discussion in line 520. Did you come across other limitations, challenges in the sessions?
In game conceptualisation (Section 2.2), it might help the reader to give a brief summary about sessions, rounds, and phases, and how they relate to the catchment / sub-catchments.
Data collection (Section 2.4) is there after the step 'modeling the game solution space' not also a step where you collect data about discussion and feed-back, which might result in adjustments of positions and solutions .? As it is an interactive process, it is collecting data and providing data, right? (so feed-back, and giving space for learning)?
An important discussion point you touch upon is how the game results translate to real life situation (line 560 and further). I wondered was this discussed in the sessions, at the end? How the game was valued, also for real-life? That could already give some initial insights?
Citation: https://doi.org/10.5194/egusphere-2023-2459-CC1 -
AC1: 'Reply on CC1', Charles Wamucii, 22 Dec 2023
CC1: 'Comment on egusphere-2023-2459', Wim Douven, 14 Dec 2023
Dear Charles and team, I enjoyed reading your manuscript. I find the topic very interesting, and it is good to have another study on serious games. Below some comments, questions, that might be helpful in finalising the manuscript. I am happy to hear you enjoyed reading the manuscript. Thank you for the positive remarks on the manuscript and constructive suggestions and comments.
Comment 1: In the section on IWRM (line 66 and further), you mention that workshops do not promote different experts to engage with wicked problems. But doesn’t that depend on how you design such workshops, like how you design serious games ..? Thank you for the question.
Response: Yes, we agree that that there are many different participatory methods developed especially to promote the engagement of 'local experts'. One of the most familiar is focussed groups, but many others are around including Dramas. Designing serious game sessions may differ from designing workshop sessions. Firstly, serious games are co-designed where both the ‘scientists’ and ‘local experts’ are involved in the development of serious games (way before actual game sessions). In the workshop sessions, the scientists’ are the most active persons, e.g. when making PowerPoint presentations, explaining an idea, reporting case studies, or simulating a model and its possible results. The participation of ‘local experts’ is relatively ‘passive’ in the workshop sessions e.g. sitting and listening to presentations made by scientists. Some workshops may divide participants into small groups for discussions, experimenting, and demonstrating workshop issues. But in most cases, the group discussions tend to have active and less active persons - in Ln 547 of the manuscript, we have argued this could be due to power differentials among participants… Workshops often are not the right way to deal with wicked problems.
There is a difference in the game sessions. Serious games provide a 'buffer' to mitigate powerplay and hierarchy, make it more easy for participants to express more freely, since it is 'just' a game. Serious games game provide a boundary object commonly not found in workshops. The ‘scientists’ are less active, and tend to play the passive role of facilitating the process (especially at the start of the game rounds), while participants are the most active persons (in small groups) right from the start to the end of the sessions. The facilitation role tends to decrease as the game progresses, and at some point ‘scientists’ become the ‘observers’ of what is happening in the different small groups of gameplay. Scientists tend to learn more this way, as opposed to when they are the main presenters in conventional workshops. The game participants are vigorous as they interact directly with ‘virtual systems’ such as Board games and also actively engage one another (e.g. disagreeing, setting rules, seeking alternative strategies, etc), the scientists do not control the actual gameplay. In Ln 542 of the manuscript, we have argued that ‘…..since each participant in the game has a role to play, they have a stake during the game. They tend to be active in pursuing their stake while focusing on their roles…’
We agree that if we design workshops similarly to game processes (including having game-like engagements during small group discussions), then designs of workshops and game sessions could both promote different experts to engage in a meaningful way. Still, in most conventional workshops the ‘scientists’ are the main ‘active persons’ while ‘local experts’ are the ‘passive’ participants. By design game sessions reverse these roles.
Comment 2: I note that the lit. review on games is overall very positive on its effects on finding joint solutions etc.., are there no limitations and challenges reported in literature ? You mention one in your discussion in line 520. Did you come across other limitations, challenges in the sessions? Thank you for the questions.
Response: This study focussed on the literature that has attempted to report on the ‘effectiveness of gaming approach’. Hence the purpose of this study was to ‘investigate these opinions/claims’ by assessing the potential role of serious gaming in strengthening stakeholder engagement. This study contributes to the debate on the ‘effectiveness of serious gaming’ including how some results contradict past studies see Ln 466-476.
We have highlighted a few limitations in this study (given the focus of the manuscript) in Ln 500 – 521. Definitely, we can probe more about the limitations and challenges of serious games (in other general/specific game settings) and incorporate them in the discussion section (limitations sub-section of this manuscript). Generally, among the limitations/challenges, the most important one is: you cannot play with people if they are in the mood to fight. So if there already is a conflict, that needs to be dealt with first. Another complication can be that some people have their reservations to engage in a game session if particular other individuals also participate - for instance, bosses with employees, in a hierarchical society.
Comment 3: In game conceptualisation (Section 2.2), it might help the reader to give a brief summary about sessions, rounds, and phases, and how they relate to the catchment / sub-catchments. Thank you for the suggestion.
Response: During this stage of game conceptualization, there was no playing the games per se, but this is the ‘setting stage’, which involved bringing all possible ideas to the table, to help in crafting a serious game that fits the context of the case study area. The ideas were gathered from past studies, through the ARDI approach, and conducting community discussions with communities in three sub-catchments. Game testing sessions are described in sub-section 2.3, where feedback was key to help in refining the final game.
Comment 4: Data collection (Section 2.4) is there after the step 'modeling the game solution space' not also a step where you collect data about discussion and feed-back, which might result in adjustments of positions and solutions .? As it is an interactive process, it is collecting data and providing data, right? (so feed-back, and giving space for learning)? Thank you for the questions.
Response: The data collection description in sub-section 2.4 focussed on data collection based on the final ENGAGE game_v1 described in Box 1 and Supplement 1. To ensure comparability of data across different game sessions, no further adjustment was made and v1 of the game was maintained (as developed) in the three sub-catchments. Yes, there is a feedback session at the end of playing the games, where participants are allowed to give their feedback and key lessons on the game sessions. The qualitative feedback was useful in crafting some of the reflections included in the discussion section e.g. in Ln 523 … the game environment allows for real-time reflection through the creation of a fictional setting and a common pool for the stakeholders to explore decisions and impacts simultaneously’……in Ln 542 …….’each participant in the game has a role to play, they have a stake during the game. They tend to be active in pursuing their stake while focusing on their roles’…….
This manuscript focusses on the ‘decisions made and qualitative aspects of gameplay’ i.e. sentiments raised during the actual gameplay. It is paramount to mention that “another manuscript (also reporting on 5 additional game sessions” focusses on the experiential learning of ENGAGE game_V1, and how stakeholders' perspectives change before and after game sessions, including the post-game individual interviews with different game participants is about to be submitted.
Comment 5: An important discussion point you touch upon is how the game results translate to real life situation (line 560 and further). I wondered was this discussed in the sessions, at the end? How the game was valued, also for real-life? That could already give some initial insights? Thank you for the questions.
Response: Yes, a conversation on translating game experience to real-life situations was a key discussion point at the end of the game sessions. There was high positivity on the usefulness of the gaming approach in promoting sustainable behaviors, and all game participants indicated they would adopt a few lessons from the game. The following are some of the direct quotes from some of the game participants:
- “I need to reduce dependency on the river, I need to construct a small dam”
- “I need to manage the land use better, considering the wet and dry seasons. I also need to construct a small dam”
- “I need to stop clearing the bushes and forest for agricultural land expansion, I better manage what I already have”
However, this manuscript recommends further follow-ups to assess the impact of serious games on strengthening stakeholders' engagement and maintaining sustainable behaviors in real life.
Follow-up data collection was done in another study (“The Manuscript on how stakeholders' perspectives change before and after serious games”), whereby game participants were traced 5-7 months after the game sessions, and most of these aspects (game experience – real life situation) have been explored.
-
AC1: 'Reply on CC1', Charles Wamucii, 22 Dec 2023
-
RC1: 'Comment on egusphere-2023-2459', Jean-Philippe Venot, 01 Mar 2024
This is a paper that investigates the use of a serious game to discuss issues of water management at watershed level in Kenya. I enjoyed reading the paper; a lot of work has gone into the design, implementation and analyze of the research. I think this type of paper reflecting on the use and scope of such participatory approaches for improving water governance is important.
That being said, I feel the paper shares what I see as a bias in the existing literature on games for sustainable natural resources management, that is, a rather positive outlook on the changes/transformations these games may trigger and a focus on their promises and potentials – that is only partly supported by what is actually described in the papers (that is what happens during “game sessions” as opposed to “in the real world”). In that specific case, I think the authors could (1) better clarify how the process they steared was fundamentally different (or not) from other more classic participatory approaches notably in relation to co-production of the tool (as opposed to a contrasting active game with passive attendance to workshop). They could also (2) engage a bit more with the risks/limitations of games that have been discussed in the literature and provide a bit more information on the (3) methodology (including the link between the game and the computer model) and (4) the mechanics of the game in the core of the paper (fez readers are likely to read the supplementary material) and (5) finally not only discuss the fact that the game seem to have enhanced communication among participants and their promises in general, but also issues related to the politics of the game themselves, that is, the extent to which the representation of the system was accepted/validated/discussed by participants and the scope for the game to change the broader rules of the game and conditions of operation of the WRUAs.
I come back to these points one by one below.
Major Comment 1. Line 66 and following: The authors attribute some of the shortcomings of IWRM to the fact that IWRM projects and policies make use of “classic” participatory approaches that would not really allow different actors to engage with the “wicked problem” at hand. In other words, participation in IWRM is not real participation. This is a well-known critique of IWRM, which I share in part, but I think there is a danger of building a strawman for the wrong reasons here, just to make the point that games do not share these shortcomings. I agree when the authors answer the previous community reviewer that games allow a different, more active, mode of engagement than classic workshop based that use power point presentation. But there are many ways to conduct workshop that can provide meaningful engagement of participants: participatory mapping, experimentation with art-based visual, etc. are some of these options.
I think the issue is less about the modalities of facilitation of workshops (and the tools used) than how multiplicity of knowledge is handled and how/who decides on the terms of the debate. Notably, who ends up deciding what the ‘wicked problem’ is. Games can actually reify the ‘wicked problem’ on the basis of an outsider/expert view; if so I would argue that the nature of engagement of participants remains as limited as in other approach or at least is constrained in major ways.What bothers me is that the phrasing of this paragraph (and many others) seems to indicate that the authors hold the view that there is ONE “wicked problem” (likely a lack of coordination between downstream and upstream users). It makes it seem as if this problem existed almost independently of the people who participate, being identified à priori by a subset of actors (though there is scope for refining the problem at hand through the use of the game itself). If that is the case, then, the game developed is likely to have very similar shortcomings than “classic” IWRM based approach.
I think the critique of IWRM approaches would be more powerful and convincing if it were to be done at the level of how are the terms of the debate/participation set and by whom (as opposed to looking at the specifics of interactions and their presumed passivity), and then to show, how this particular study did something different, hence created the room for another type of engagement by the people who participated in the game session. The point here is to pay a bit more attention to the politics that framed the entire research process and operated silently in the background as opposed to take the dynamism of the game sessions as a sign of pro-active engagement and changes to come in the watershed. The authors may want to look at https://hess.copernicus.org/preprints/hess-2023-164/ currently under review in the same journal. This paper reflects on some of those issues around the practices of hydrological modeling not games, but I think there are many similariries.
I do not doubt the authors have done something quite different from “classic passive workshops”; my qualm is with the way authors describe the differences. Instead of making clear what the differences were, the way it is written points to fundamental similarities in the assumptions underlying the approach (which may not have been there). I feel clarifying this point through some reformulation is important and would strengthen the argument of the paper.
Major Comment 2. Line 93 and following. The way to depicts serious games and what they can help achieve is overwhelmingly positive. I think this reflects rather well the existing literature on serious games, most of which is actually written by people involved in their design and implementation – which may make them a bit less likely to adopt a critical stance towards these tools. Yet, I think the authors could engage with some of the limits of games that have been discussed in the literature, notably by members of the ComMod collective they are likely to know. There is some work in relation to conflicts and existing relations of domination (Barnaud & Van Paassen, 2013; Bécu et al., 2008; Mathevet et al., 2014), on the roles and postures of facilitators and modelers (Barreteau et al., 2003; Jonsson et al., 2007), or the risk of manipulation (Barnaud et al., 2008; Halbe et al., 2018). A recent overview recognizes ‘‘potential biases” (Barreteau et al., 2021). See also (Jonsson et al., 2007) on whether games are “truly participatory”. Also, the fact that games simplify complex problems (line 102) is presented as a positive thing (I suspect in relation to legibility of the said problem). This may be the case, but it also raises a lot of questions: who decides what is represented and what is not, what are the implications of those choices… In other words, what are the politics of the game itself? These questions ought to be at least mentioned I think.
Major Comment 3. I feel there is a need to clarify the methodology. Notably, in the first paragraph it is not clear to me what the “solution space” of the game (what are ‘responses’ to rules?). It is also not clear what the authors mean by “overall performance of the game results” (how is performance assessed, is it in relation to water availability/sharing? In relation to how active participation was…)? In relation to that, section 2.4.3 discusses a modeling of the ‘game solution space’ and mentions a 1000 runs but what are those runs? I assume the authors used both a physical game and a computer version of it and that they used the later a 1000 times generating random actions; this to situate the results of the specific game sessions in the “realm of possible”. But I can only assume this: the relations between game and model need to be made explicit and the authors need to explain, why the use of the model was useful, what did it allow to do?
Major Comment 4. A lot of key information on the mechanics of the game (including how the connections between actors and water flows are conceptualized) are included in the supplementary material. I think only few readers are likely to read the supplementary material and it might be a good idea if the authors extended the presentation of the game in the core of the paper slightly. I’m also not sure the game should be presented in a “box”. Normal section numbering would do. In terms of presenting the game, I think it would be interesting to have a sub-section describing the board and the players, another one describing briefly the different action/decision each actor can make and how this impact water flows (at a conceptual level; not entering in the details of the calibration), and another one on the “typical” process followed during each session – this is already in large part in the box. I think the graph on the system dynamic modeling could support this presentation of the game, as opposed to be presented in relation to the “solution space” of the game. The supplementary file can then focus on the calibration aspect. Presenting the game as such would also allow having part of the discussion engaging with the key stage of game validation that is not really described at the moment (what hypothesis the authors made were validated, which one were not…? for instance deciding that water dries up from downstream when people abstract water from upstream is a strong choice. I would be curious to know if that triggered some discussion also because, materially, it involves taking a marble from one area of the board to put it in the other one, which is not really how water abstraction works in the real world.)
Major comment 5. The discussion focuses on what has happened during the game session and it is important. The game sessions are likely to have improved communication and cross learning among participants but to which extent did it actually provide ways to solve the structural issues faced by WRUA that are identified in line 188 (weak enforcement of policies/laws, water abstraction regulations, water metering requirements, protection of riparian corridors/forested areas, etc.)? Such structural issues are likely to relate to the institutional strength of the WUAs, how legitimate it is vis-à-vis other actors, the (human) and financial means it has… Even if it is not the objective of this paper to assess the scope of the game to change things in the real world, I think it is important for the authors to engage in a discussion around the promises and potential of games in relation to these structural hurdles.
Most of the literature on games tend to focus on the design process and on what is happening during game sessions, and that’s why most of the literature is positive: there is a lot of interesting things happening, some of which do have transformative potential. But in doing so, the game and the overall research process is also “removed” from the overall political-economic context in which it is implemented while this context is pivotal for the promises of the game to actually materialize or not. While describing the overall political-economy that influences water management in the case study area is beyond the scope of this paper, I think it is important the authors reflect a bit on it and how this might influence the future of the game outcomes – to avoid falling in the trap of highlighting promises that may never materialize.
I also have some specific comments:
Line 21 (Abstract): What do the authors mean by “shared understanding”, and understanding of what exactly? Could it be clarified?
Line 27 (abstract): This sentence seems to point to the fact that the objective of the study was to assess the potential of the game to strengthen stakeholder engagement “next” (that is, after the game sessions were held). I do not think the paper demonstrates this. It demonstrates rather convincingly that there was active engagement during the game session but does not engage with whether or not this translated in strengthened engagement outside of the game session. I understand from the exchange with the community member who commented on the paper that this is not the point of this particular paper but then, maybe this sentence needs to be rephrased to indicate that what is being described in the dynamics of engagement during gaming session. Similarly, some clarity may be needed on line 125.
Line 47. The term “wicked problems” was first coined by Horst Rittel and Melvin Webber in a 1973 paper, which would be good to mention.
Line 54 to 56: It is not clear how the 5 interacting phases of public debate had a bearing on the research conducted. Why do the authors refer to this work, and how useful is it to understand the process of designing and implementing game sessions? This is clarified around line 95. Consider slight restructuring on when this is mentioned so as to avoid repetitions?
Line 58 and following: I would not call IWRM an “approach”. In my view it consists rather in a policy and discursive model than an approach per se.
Line 63: can the authors briefly say what successes were actually achieved? In what terms was implementation successful and in which contexts?
Line 107: improved efficiency of what?
Line 101: what does “this” relate to?
Line 112 and following: I am not sure what those sentences are meant to bring to the paper. They look as quite general statements in relation to games (what others have said on games), but it is not clear how much of this is actually being used in this particular study. Maybe they do not belong here but should come earlier in the paragraph? I have a similar remark on communication: are the two sentences on that specific topic meant to discuss game “in general” or has this topic of communication during game played been a specific entry point of analysis of what has happened in the game session organized in this study?
Line 132: what do the authors mean by “sentiment” and how did they assess it. What do they mean by “active” participation and how was it assessed.
Line 145 and following: why does the discussion focuses on “substractive dynamics” only and not on “constructive dynamics”?
Line 160 and following: how many WRUA are there in the sub basin/case study area? One for each sub-river?
Line 177and following: This points to my first major comment. How was this suite of problems identified and by whom? And more specifically, have the people who participated to the game sessions been involved in the definition of these problems. This needs to be clarified as it is largely from this that one can assess to which extent the process followed is fundamentally different from classic “participatory activities” implemented under IWRM processes.References:
Barnaud, C., Trébuil, G., Promburom, P., & Bousquet, F. (2008). La modélisation d’accompagnement pour une gestion concertée des ressources renouvelables en
Thaïlande. Economie Rurale, 303–304–305, 39–59.Barnaud, C., & Van Paassen, A. (2013). Equity, power games, and legitimacy: Dilemmas of participatory natural resource management. Ecology and Society,
18(2), 21. https://doi.org/10.5751/ES-05459-180221.Barreteau, O., Antona, M., d’Aquino, P., Aubert, S., Boissau, S., Bousquet, F., Daré, W., Etienne, M., Le Page, C., Mathevet, R., Trébuil, G., & Weber, J. (2003). Our
companion modelling approach. Journal of Artificial Societies and Social Simulation, 6(1).Barreteau, O., Abrami, G., Bonte, B., Bousquet, F., & Mathevet, R. (2021). Serious Games. In R. Biggs, A. de Vos, R. Preiser, H. Clements, K. Maciejewski, & M.
Schlüter (Eds.), The Routledge handbook of research methods for social- ecological systems (pp. 176–188). London & New York: Routledge.Bécu, N., Neef, A., Schreinemachers, P., & Sangkapitux, C. (2008). Participatory computer simulation to support collective decision-making: Potential and
limits of stakeholder involvement. Land Use Policy, 25(4), 498–509. https://doi. org/10.1016/j.landusepol.2007.11.002.Halbe, J., Pahl-Wostl, C., & Adamowski, J. (2018). A methodological framework to support the initiation, design and institutionalization of participatory modeling
processes in water resources management. Journal of Hydrology, 556, 701–716. https://doi.org/10.1016/j.jhydrol.2017.09.024.Jonsson, A., Andersson, L., Alkan-Olsson, J., & Arheimer, B. (2007). How participatory can participatory modeling be? Degrees of influence of stakeholder and expert
perspectives in six dimensions of participatory modeling. Water Science and Technology, 56(1), 207–214. https://doi.org/10.2166/wst.2007.453Mathevet, R., Antona, M., Barnaud, C., Fourage, C., Trébuil, G., & Aubert, S. (2014). Contexts and dependencies in the commod processes. In M. Etienne (Ed.),
Companion Modelling: A participatory approach to support sustainable development (pp. 103–125). Dordrecht: Springer.Venot, JP.; Jensen CB.; Delay E. et Daré, W. (2022). Mosaic glimpses: Serious games, generous constraints, and sustainable futures in Kandal, Cambodia. World Development 151 (2022):105779.
Citation: https://doi.org/10.5194/egusphere-2023-2459-RC1 -
AC2: 'Acknowledgement of referee comments:', Charles Wamucii, 05 Mar 2024
I am happy to hear you enjoyed reading the manuscript. Thank you for the remarks on the manuscript and constructive suggestions and comments. I acknowledge receipt of the following issues as the major comments:
- Clarifying how the process was fundamentally different (or not) from other more classic participatory approaches notably in relation to co-production of the tool (as opposed to a contrasting active game with passive attendance to workshop).
- Expand the discussion on the risks/limitations of games that have been discussed in the literature.
- Provide more information on the methodology (including the link between the game and the computer model)
- Expand the game mechanics of the game in the core of the paper (most details are captured in the supplementary material)
- Expand discussion beyond enhanced communication during gameplay to also include game politics.
I will come back with detailed responses to each of these comments and suggestions as soon as possible. I am currently preparing for my PhD defence, which will be held next week on the 13th of March 2024, at 11:00 AM (Amsterdam time) at Wageningen University. This will also be broadcast live here: (https://wur.yuja.com/). Perhaps some of these issues may come up during my PhD defence.
Regards,
Citation: https://doi.org/10.5194/egusphere-2023-2459-AC2 -
AC3: 'Reply on RC1', Charles Wamucii, 25 Mar 2024
Dear referee,
Thank you for the acknowledgment of the relevance of our research exploring the potential of participatory approaches such as serious games for improving water resources
management and governance. In the attachment below, find our responses and how we plan to address your major five comments and specific comments.Regards,
Charles, on behalf of Co-authors
-
AC2: 'Acknowledgement of referee comments:', Charles Wamucii, 05 Mar 2024
-
EC1: 'Comment on egusphere-2023-2459', Pieter van der Zaag, 07 Mar 2024
I herewith would like to note that the Community Comment by Wim Douven (CC1) should be considered to be a Referee Comment, as I had invited and nominated Wim Douven as a referee for this article. For some reason his comment was labelled as a CC and not as an RC.
Citation: https://doi.org/10.5194/egusphere-2023-2459-EC1
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 306 | 123 | 34 | 463 | 48 | 29 | 25 |
- HTML: 306
- PDF: 123
- XML: 34
- Total: 463
- Supplement: 48
- BibTeX: 29
- EndNote: 25
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Charles Nduhiu Wamucii
Pieter R. van Oel
Adriaan J. Teuling
Arend Ligtenberg
John Mwangi Gathenya
Gert Jan Hofstede
Meine van Noordwijk
Erika N. Speelman
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2894 KB) - Metadata XML
-
Supplement
(1259 KB) - BibTeX
- EndNote
- Final revised paper