the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 10 Jun 2024
Designing and evaluating a public engagement activity about sea level rise
Abstract. In this paper, we describe the design process of a public engagement activity about sea level rise aimed at young adults aged 16 to 25, intended to enhance participants’ response efficacy and perceived relevance. We conducted the activity at multiple occasions and performed a statistical analysis of the impact measurement among 117 participants. Based on the analysis and observations, we conclude that the activity resonated well with our target audience, regardless of their level of science capital, suggesting that a design study approach is well-suited for the development of similar activities.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(659 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(659 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-1649', Rebecca Priestley, 21 Jul 2024
This paper describes a public engagement activity – a game – about sea level rise, aimed at 16–25-year-olds in the Netherlands. The paper shares the design process of the activity through a prototyping stage to development of the final game, and then reports on results of a questionnaire to assess the impact of the game with participants. Public engagement with sea level rise is an important area of research and it is interesting and useful to have insight into the design process that led to the development of this activity, as well as the impact of the game on participants. This will be a valuable paper once some things have been addressed to ensure that the paper is (1) engaging with additional relevant scholarly activity and (2) communicating more clearly with readers.
First, I think the paper should engage with the scholarship on serious games, particularly around coastal adaptation, and situate this study in relation to this work. Here are some papers to look at:
Flood et al, 2018, ‘Adaptive and interactive climate futures: systematic review of 'serious games' for engagement and decision-making’, Environ. Res. Lett. 13 063005, https://iopscience.iop.org/article/10.1088/1748-9326/aac1c6
Lawrence, J. and Haasnoot, M. (2017). ‘What it took to catalyse a transition towards adaptive pathways planning to address climate change uncertainty.’ Environmental Science and Policy. http://dx.doi.org/10.1016/j.envsci.2016.12.003Â
Yang, Wei and Harrison, Sarah and Blackett, Paula and Allison, Andrew, ‘An explorative analysis of gameplay data based on a serious game of climate adaptation in Aotearoa New Zealand’ SSRN, (2024). Available at http://dx.doi.org/10.2139/ssrn.4818597
Lawrence Judy , Stephens Scott , Blackett Paula , Bell Robert G. , Priestley Rebecca, ‘Climate Services Transformed: Decision-Making Practice for the Coast in a Changing Climate’ Frontiers in Marine Science, v8, 2021
https://www.frontiersin.org/journals/marine-science/articles/10.3389/fmars.2021.703902/full
Â
Other overriding issues are:
- there is too much emphasis in the paper on the prototype, and not enough on the final product. I found it hard to understand the game until I read the supplementary material, and I suggest that some relevant information from the supplementary material be included in the paper
- the order in which information is revealed is not always helpful – some things that would have been useful up front are not revealed until later in the paper
- the graphics need better captions, headings and contextualizing
- some language needs to be revised for clarity
Â
Page specific comments follow:
Abstract: the phrase ‘response efficacy and perceived relevance’ is not conveying much useful information here, suggest revise
Line 55: In discussing science capital you should also cite Archer et al 2015 which is in the list of references but is not cited (I suggest a thorough check to ensure papers cited are in the list of references and vice versa)
Line 88: what do you mean by ‘people like’ the participants? Maybe there’s another way of phrasing this, referring more specifically to the demographics of this group?
Line 116: 2080 was chosen as ‘most of the young adult participants will still be alive by then’ which is a good explanation of why a later date, 2100 for example, was not chosen, but why not an earlier date, say 2050 or 2060 for example? A simple explanation would help.
Page 6, figure 1: There’s not enough information for this figure to be useful. What do the numbers represent? What are A and B? For this to be useful, the reader needs to know what the six dilemmas are, and how the questions asked relate to this figure. Telling us what the dilemmas are would also help section 2.3 on page 5. If space is an issue, then knowing the specific dilemmas presented to players would be more useful than knowing the options chosen in the prototyping stage.
Page 6, 170: was there really a team with only one player? If so, why, and did that impact on how that game went?
Figure 3, p7: For figure 3 to be useful we need to know what ‘low’ ‘medium’ and ‘high’ mean
Figure 4, p8: Again, we need some information about what ‘low’ ‘medium’ and ‘high’ mean
Section 2.5: If there is space, it would be useful to have the final version of the game board, along with the questionnaire, in the paper itself rather than in the supplementary papers. At the moment, there’s a real emphasis on the prototype – which is good to learn about the design process – but not enough on the game itself
Line 288: this phrase ‘response efficacy and perceived relevance’ is not conveying a lot of information, I suggest rephrase
Line 330: this line refers to ‘low sea level, high solution level’ etc – if this is what the low/low, low/medium’ etc was referring to in the graphic earlier, this information should be included earlier in the paper
Line 379: This specific information about the questionnaire would have been useful earlier
Looking at the supplementary papers gave me a much better understanding of the game. If there is space, I would suggest that some specific information about the dilemmas, and the future scenarios, be included in the main paper.
Citation: https://doi.org/10.5194/egusphere-2024-1649-RC1 - AC1: 'Reply on RC1', Nieske Vergunst, 29 Oct 2024
-
RC2: 'Comment on egusphere-2024-1649', Sam Illingworth, 01 Oct 2024
Overview
This is an interesting and valuable paper on the design and evaluation of a public engagement activity focusing on sea level rise. The manuscript is very well-written, engaging, and easy to follow. It presents a thoughtful approach to addressing the important topic of climate change, specifically in relation to sea level rise in the Netherlands. There are, however, a few areas where further clarification or expansion could enhance the overall impact of the paper, particularly around the framing of the audience, some methodological choices, and the conclusions.
Major Comments
- Clarifying the audience in the abstract. While the abstract is well-crafted, it would benefit from more specific details about the target audience, particularly the Netherlands context. Additionally, it would be helpful to mention the recommendations from your findings and how (or whether) these could be broadly implemented elsewhere, along with any potential limitations.
- Section 1.1: Climate change and sea level rise. The introduction does an effective job of setting the scene, but the rapid transitions between statistics for different countries could be more cohesively presented. Consider linking these statistics together more fluidly and clearly highlighting why the focus of this project is on the Netherlands, addressing the specific challenges faced in this region.
- Clarifying 'the public' in Section 1.2. Throughout the manuscript, there is frequent reference to 'the public.' It would be more accurate to acknowledge the existence of multiple publics, as there is no homogeneous 'general public.' I recommend reworking this terminology throughout. For further context, I discuss this concept in my paper 'A spectrum of geoscience communication: from dissemination to participation' (Illingworth, 2023), but feel free to reference other relevant literature instead. The key point is to ensure you are capturing the diversity of audiences.
- Rationale for choosing a board game in Section 2. The design process is excellent and will be very useful for others working in the field. It would strengthen this section if you explicitly explained why a board game was chosen as the public engagement format over other possible methods. Additionally, a discussion of the strengths and limitations of this approach, as well as how they were realised in practice, would be valuable.
- Interpreting statistical analysis in Section 4. The statistical analysis is well-explained and contextualised, with observational data effectively supporting the analysis. However, I would have liked to see more interpretation of what these results mean, particularly in relation to the ‘success’ of the game and its potential implications for future development. Expanding on how useful these results were in informing your conclusions would add depth to this section.
- Audience biases. It would be beneficial to reflect on the potential biases or limitations of the audience sampled, for example, acknowledging that participants attending an open day for prospective Bachelor students may have higher levels of science capital and interest compared to other publics. This consideration could help further contextualise your findings.
- Conclusions and recommendations. While Section 5.1 on limitations is particularly strong, the conclusions could be more robust. I would recommend strengthening the final section by tying it back more explicitly to the initial research questions. Including clear, actionable recommendations for other researchers or practitioners based on your work would also be a valuable addition. These recommendations could also be incorporated into the abstract to give the reader a clearer sense of the broader applicability of your findings.
- Ethical considerations. The ethical statement provided is excellent. However, I suggest integrating aspects of this statement into the main body of the text in Section 2, particularly a discussion of the risks and benefits identified during the ethical review process conducted by Utrecht University. This will provide a more holistic view of the study’s ethical considerations.
Â
Minor Comments
- Subheadings in Section 1. While Section 1 provides a very solid introduction, the number of subheadings may make it feel somewhat fragmented. Consider consolidating some of these sub-sections to provide a smoother reading experience.
- Captions. The captions are generally clear but could be expanded so that the figures can be interpreted independently of the main text.
- Figure 5. The quality of Figure 5 is somewhat blurry. It might be more effective to present the information as text within the main body of the paper rather than as a figure.
References
Illingworth, S.: A spectrum of geoscience communication: from dissemination to participation, Geosci. Commun., 6, 131–139, https://doi.org/10.5194/gc-6-131-2023 , 2023.
Â
Citation: https://doi.org/10.5194/egusphere-2024-1649-RC2 - AC2: 'Reply on RC2', Nieske Vergunst, 29 Oct 2024
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-1649', Rebecca Priestley, 21 Jul 2024
This paper describes a public engagement activity – a game – about sea level rise, aimed at 16–25-year-olds in the Netherlands. The paper shares the design process of the activity through a prototyping stage to development of the final game, and then reports on results of a questionnaire to assess the impact of the game with participants. Public engagement with sea level rise is an important area of research and it is interesting and useful to have insight into the design process that led to the development of this activity, as well as the impact of the game on participants. This will be a valuable paper once some things have been addressed to ensure that the paper is (1) engaging with additional relevant scholarly activity and (2) communicating more clearly with readers.
First, I think the paper should engage with the scholarship on serious games, particularly around coastal adaptation, and situate this study in relation to this work. Here are some papers to look at:
Flood et al, 2018, ‘Adaptive and interactive climate futures: systematic review of 'serious games' for engagement and decision-making’, Environ. Res. Lett. 13 063005, https://iopscience.iop.org/article/10.1088/1748-9326/aac1c6
Lawrence, J. and Haasnoot, M. (2017). ‘What it took to catalyse a transition towards adaptive pathways planning to address climate change uncertainty.’ Environmental Science and Policy. http://dx.doi.org/10.1016/j.envsci.2016.12.003Â
Yang, Wei and Harrison, Sarah and Blackett, Paula and Allison, Andrew, ‘An explorative analysis of gameplay data based on a serious game of climate adaptation in Aotearoa New Zealand’ SSRN, (2024). Available at http://dx.doi.org/10.2139/ssrn.4818597
Lawrence Judy , Stephens Scott , Blackett Paula , Bell Robert G. , Priestley Rebecca, ‘Climate Services Transformed: Decision-Making Practice for the Coast in a Changing Climate’ Frontiers in Marine Science, v8, 2021
https://www.frontiersin.org/journals/marine-science/articles/10.3389/fmars.2021.703902/full
Â
Other overriding issues are:
- there is too much emphasis in the paper on the prototype, and not enough on the final product. I found it hard to understand the game until I read the supplementary material, and I suggest that some relevant information from the supplementary material be included in the paper
- the order in which information is revealed is not always helpful – some things that would have been useful up front are not revealed until later in the paper
- the graphics need better captions, headings and contextualizing
- some language needs to be revised for clarity
Â
Page specific comments follow:
Abstract: the phrase ‘response efficacy and perceived relevance’ is not conveying much useful information here, suggest revise
Line 55: In discussing science capital you should also cite Archer et al 2015 which is in the list of references but is not cited (I suggest a thorough check to ensure papers cited are in the list of references and vice versa)
Line 88: what do you mean by ‘people like’ the participants? Maybe there’s another way of phrasing this, referring more specifically to the demographics of this group?
Line 116: 2080 was chosen as ‘most of the young adult participants will still be alive by then’ which is a good explanation of why a later date, 2100 for example, was not chosen, but why not an earlier date, say 2050 or 2060 for example? A simple explanation would help.
Page 6, figure 1: There’s not enough information for this figure to be useful. What do the numbers represent? What are A and B? For this to be useful, the reader needs to know what the six dilemmas are, and how the questions asked relate to this figure. Telling us what the dilemmas are would also help section 2.3 on page 5. If space is an issue, then knowing the specific dilemmas presented to players would be more useful than knowing the options chosen in the prototyping stage.
Page 6, 170: was there really a team with only one player? If so, why, and did that impact on how that game went?
Figure 3, p7: For figure 3 to be useful we need to know what ‘low’ ‘medium’ and ‘high’ mean
Figure 4, p8: Again, we need some information about what ‘low’ ‘medium’ and ‘high’ mean
Section 2.5: If there is space, it would be useful to have the final version of the game board, along with the questionnaire, in the paper itself rather than in the supplementary papers. At the moment, there’s a real emphasis on the prototype – which is good to learn about the design process – but not enough on the game itself
Line 288: this phrase ‘response efficacy and perceived relevance’ is not conveying a lot of information, I suggest rephrase
Line 330: this line refers to ‘low sea level, high solution level’ etc – if this is what the low/low, low/medium’ etc was referring to in the graphic earlier, this information should be included earlier in the paper
Line 379: This specific information about the questionnaire would have been useful earlier
Looking at the supplementary papers gave me a much better understanding of the game. If there is space, I would suggest that some specific information about the dilemmas, and the future scenarios, be included in the main paper.
Citation: https://doi.org/10.5194/egusphere-2024-1649-RC1 - AC1: 'Reply on RC1', Nieske Vergunst, 29 Oct 2024
-
RC2: 'Comment on egusphere-2024-1649', Sam Illingworth, 01 Oct 2024
Overview
This is an interesting and valuable paper on the design and evaluation of a public engagement activity focusing on sea level rise. The manuscript is very well-written, engaging, and easy to follow. It presents a thoughtful approach to addressing the important topic of climate change, specifically in relation to sea level rise in the Netherlands. There are, however, a few areas where further clarification or expansion could enhance the overall impact of the paper, particularly around the framing of the audience, some methodological choices, and the conclusions.
Major Comments
- Clarifying the audience in the abstract. While the abstract is well-crafted, it would benefit from more specific details about the target audience, particularly the Netherlands context. Additionally, it would be helpful to mention the recommendations from your findings and how (or whether) these could be broadly implemented elsewhere, along with any potential limitations.
- Section 1.1: Climate change and sea level rise. The introduction does an effective job of setting the scene, but the rapid transitions between statistics for different countries could be more cohesively presented. Consider linking these statistics together more fluidly and clearly highlighting why the focus of this project is on the Netherlands, addressing the specific challenges faced in this region.
- Clarifying 'the public' in Section 1.2. Throughout the manuscript, there is frequent reference to 'the public.' It would be more accurate to acknowledge the existence of multiple publics, as there is no homogeneous 'general public.' I recommend reworking this terminology throughout. For further context, I discuss this concept in my paper 'A spectrum of geoscience communication: from dissemination to participation' (Illingworth, 2023), but feel free to reference other relevant literature instead. The key point is to ensure you are capturing the diversity of audiences.
- Rationale for choosing a board game in Section 2. The design process is excellent and will be very useful for others working in the field. It would strengthen this section if you explicitly explained why a board game was chosen as the public engagement format over other possible methods. Additionally, a discussion of the strengths and limitations of this approach, as well as how they were realised in practice, would be valuable.
- Interpreting statistical analysis in Section 4. The statistical analysis is well-explained and contextualised, with observational data effectively supporting the analysis. However, I would have liked to see more interpretation of what these results mean, particularly in relation to the ‘success’ of the game and its potential implications for future development. Expanding on how useful these results were in informing your conclusions would add depth to this section.
- Audience biases. It would be beneficial to reflect on the potential biases or limitations of the audience sampled, for example, acknowledging that participants attending an open day for prospective Bachelor students may have higher levels of science capital and interest compared to other publics. This consideration could help further contextualise your findings.
- Conclusions and recommendations. While Section 5.1 on limitations is particularly strong, the conclusions could be more robust. I would recommend strengthening the final section by tying it back more explicitly to the initial research questions. Including clear, actionable recommendations for other researchers or practitioners based on your work would also be a valuable addition. These recommendations could also be incorporated into the abstract to give the reader a clearer sense of the broader applicability of your findings.
- Ethical considerations. The ethical statement provided is excellent. However, I suggest integrating aspects of this statement into the main body of the text in Section 2, particularly a discussion of the risks and benefits identified during the ethical review process conducted by Utrecht University. This will provide a more holistic view of the study’s ethical considerations.
Â
Minor Comments
- Subheadings in Section 1. While Section 1 provides a very solid introduction, the number of subheadings may make it feel somewhat fragmented. Consider consolidating some of these sub-sections to provide a smoother reading experience.
- Captions. The captions are generally clear but could be expanded so that the figures can be interpreted independently of the main text.
- Figure 5. The quality of Figure 5 is somewhat blurry. It might be more effective to present the information as text within the main body of the paper rather than as a figure.
References
Illingworth, S.: A spectrum of geoscience communication: from dissemination to participation, Geosci. Commun., 6, 131–139, https://doi.org/10.5194/gc-6-131-2023 , 2023.
Â
Citation: https://doi.org/10.5194/egusphere-2024-1649-RC2 - AC2: 'Reply on RC2', Nieske Vergunst, 29 Oct 2024
Peer review completion
Journal article(s) based on this preprint
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 275 | 80 | 295 | 650 | 22 | 18 |
- HTML: 275
- PDF: 80
- XML: 295
- Total: 650
- BibTeX: 22
- EndNote: 18
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Tugce Varol
Erik van Sebille
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(659 KB) - Metadata XML