the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 13 Mar 2024
The (non)effect of personalization in climate texts on credibility of climate scientists
Abstract. How we communicate about climate change affects how others think, feel and act. Therefore, the way climate scientists formulate messages is important. In this study, we assess the effect of personalization, operationalized as writing in a conversational style, as previously done by Ginns and Fraser (2010), and perceived credibility of climate scientists. We exposed hundred participants aged between 18 and 35 to three conditions of a text on the climate impact of train versus plane travel, with varying degree of personalization, and assessed the outcome in their attitude (specifically interest and opinion) towards sustainable travel, as well as the perceived credibility of the climate scientist who wrote the text. Results show that there is a small effect in the degree of happiness after reading the different texts, but little other effects. Our main conclusion is that, although personalization may be well received by readers, it may not be the best mode to influence the attitudes of readers towards sustainable travel, nor how readers come to perceive climate scientists' credibility.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(1157 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(1157 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-543', Sam Illingworth, 20 May 2024
Overview
The manuscript presents an interesting and relevant subject that is a good fit for Geoscience Communication. The study explores the effects of personalization in climate communication, specifically looking at the credibility of climate scientists. Overall, the manuscript is well-structured and easy to read. However, there are several areas that require further development to enhance the clarity and impact of the work.
Major Comments
- Introduction expansion. The introduction needs expanding, particularly around the IMPACTLAB instrument. It is currently unclear what this instrument is, how it is implemented, and its relevant strengths and weaknesses compared to other approaches. This additional detail will provide a stronger foundation for your study.
- Section 1.2 on personalization. While Section 1.2 on Personalization is good, it would benefit from a more thorough discussion of how other publics, such as lobbyists and politicians, might use this rhetorical technique to undermine the work of scientists. Additionally, exploring how scientists and science communicators can learn from this approach would be valuable. Including examples and references where this approach has worked effectively (or not) in contexts like vaccinations and anti-smoking campaigns would strengthen this section.
- Selection of KH article. Clarify how the KH article was chosen in terms of topic. Discuss the implications of this choice for the results. For instance, consider whether topics such as ocean acidification or sea level rises affecting Indigenous communities might have yielded different outcomes.
- Translation and personalization. The issue of translation requires further exploration. Discuss how personalization might differ in Dutch compared to English, and what this implies for other languages. This will help in understanding the broader applicability of your findings.
- SurveySwap details. Provide more information about SurveySwap. Explain what it is and how it works in terms of reliability. This will help readers assess the robustness of your methodology.
- Consent forms and ethical considerations. Expand on the discussion around consent forms. What risks and benefits did you consider and communicate to participants? How was the data stored securely? Include more details on these aspects. Additionally, bring some of the 'Ethical statement' (line 325) into the main body of the text, such as the classification of the study as low risk, which negated the need for further ethical review or privacy assessment.
- Determining personalization levels. Explain how you determined if a text was slightly or highly personalized. Acknowledge the potential subjectivity here and discuss how this assessment could be repeated reliably.
- Title scope. The study focuses on one example – sustainable travel. The current title, "The (non)effect of personalization in climate texts on the credibility of climate scientists," is too broad. Consider revising the title to better reflect the scope of the study, such as "The (non)effect of personalization in climate texts on the credibility of climate scientists: A case study on sustainable travel."
- The conclusions currently read more like a summary of the article. Expand this section to include the significance of the work and its implications for future studies. This will provide a more impactful closing to your manuscript.
Minor Comments
- Abstract. The abstract is well-crafted and easy to digest. No changes are necessary here.
- Figures and captions. The figures are clear, well-produced, and effectively support the text. However, the captions could benefit from a little more detail so they can be understood independently of the main body of the article.
- Section 2.5 on quantitative statistics. Section 2.5 on quantitative statistics for the control is sound and well-explained but does not integrate smoothly with the preceding and following sections. Improving the transitions will enhance the overall flow of the manuscript.
- Statistical tests explanation. Every time you introduce a statistical test or method, such as the Cronbach-Alpha score, provide an explanation of what the test is and what certain scores mean. This will help the reader understand the methodology better and make the manuscript more accessible.
- Discussion and Research Questions. The discussion would benefit from clearer presentation of the Research Questions and their integration throughout the text. This will help in drawing direct connections between your findings and the questions posed.
- Fair discussion of limitations. The discussion of limitations is very fair and well done. This transparency strengthens the credibility of your study.
Citation: https://doi.org/10.5194/egusphere-2024-543-RC1 -
AC1: 'Reply on RC1', Erik van Sebille, 28 May 2024
Dear Sam, tank you very much for these useful comments. In the attached pdf, we sketch how we will take these into account in a revised version of the manuscript. We will, however, wait until we also receive the comments of reviewer 2 before we upload that revision.
-
RC2: 'Comment on egusphere-2024-543', Louise Arnal, 14 Jun 2024
In this manuscript, the authors examine how personalizing a science text about travel modes and climate change influences participants' attitudes towards climate change and their perception of the writer's credibility. The study reveals that personalization had some effect on participants' emotions but did not significantly alter their attitudes or perceptions of the scientist's credibility. Overall, I found the manuscript very engaging and a thoughtful contribution to Geoscience Communication and the literature. However, adding more nuances to the results and discussion sections would enhance its depth. Below are my comments, which I hope will be helpful to revise your manuscript for publication.
General comments:
- It would be beneficial to expand the discussion on scientists expressing their opinions. While you introduce this concept on lines 58-70, revisiting it in the discussion with additional insights would be valuable. Often, scientists are guided by their institutions to avoid taking official stances. Given the urgency of the climate crisis, should scientists feel more comfortable advocating, and should we train future scientists to do this more effectively? Additionally, should workplaces support this type of communication, or should it be left to climate communication experts?
- Why was the personalization conducted in two steps instead of comparing a single personalized version to the original? This approach would have increased the sample sizes for each group. Please provide an explanation in the manuscript to clarify the rationale behind this methodology.
- Language use, including expressions and humour, is highly dependent on cultural context. A personalized text effective in one language would require careful translation to capture these nuances for different populations, whereas a more expository text might not need the same level of adjustment. Could you discuss this consideration in the manuscript?
- I couldn’t find the appendix.
- Much of the content in section 2.4 seems more appropriate for the results section. Please consider moving this material to a sub-section of the results.
- The results section could benefit from more detailed description in places. Firstly, some results indicate statistical significance between groups without specifying the direction of the effect. For example, on lines 211-213, we know that there is a difference in responses regarding the writer's perceived credibility, but it is unclear how prior likelihood to fly or science capital and trust (SCT) influence these responses. Similarly, this applies to lines 184-186 and 234-236 (please review for other instances). Secondly, even when results are not statistically significant, I think that it would still be useful to describe them qualitatively, clearly distinguishing between statistically significant and non-significant findings. For instance, regarding SCT on responses to control questions (Fig. 3), although not statistically significant, there is a qualitative difference where a higher percentage of participants with higher SCT found the texts professional/formal. Despite the sample size limiting the detection of small differences as you mention in the discussion, these can still be qualitatively appreciated. Adding such qualitative descriptions throughout the manuscript would highlight nuances not currently captured in the text but shown in the figures. Please consider discussing such subtleties, future research directions, and hypotheses for further studies with larger samples to explore these minimal effects more thoroughly.
- I find the question in section 3.3 about whether the text aims to persuade or inform intriguing, but it should be introduced earlier in the manuscript for better context. Additionally, the impact of participants’ SCT on their perception of the writer's impartiality (second question) is not discussed, warranting further exploration in the discussion section.
- Participants' prior knowledge (and possibly trust) of the KH platform and the writer's name could have influenced the results. Was this information provided to participants before they answered the questions? Clarifying this could help understand its impact.
Specific comments:
-L18: It would be useful to provide a brief description of the word ”attitude” in this context as it is very broad, perhaps using the text in between parentheses on L76.
-L20-22: This sentence talks about strengthening the link between climate information and action. There needs to be another sentence before that to discuss the existence of that link. Could you cite some literature that discusses the impact of (climate) information for changing attitudes and possibly actions?
-L25-31: I find that this more methodological piece would fit better later in the introduction, once the research questions have been introduced. Where it stands, it disrupts a bit the flow of a very interesting and broader introduction.
-L32: There is no section 1.1.
-L50-51: You already mentioned that attitudes affect climate action in the first sentences of the introduction. Perhaps rephrase this sentence slightly to incorporate it with the previous sentence and remove the repetition.
-L99-100: Could you include the full second sentence of the text so that it is a bit easier to interpret the meaning and changes made? Same for conditions 2 and 3.
-L116: Did all participants live in The Netherlands?
-L117-118: Why was the focus on young adults specifically? Please clarify this choice in the manuscript.
-L121: Does ”M” in the parentheses refer to mean or median? This could be written out for clarity and to match the format in section 2.4.1.
-L122: What are the results from the educational level? It is only mentioned in the discussion.
-L123: Will the questions be available in the appendix?
-L127: This is the first time that the word ”behaviour” is used in the manuscript. It would be useful to have a brief description of what you mean, and how it situates itself within the attitude, opinion, and action pieces you already mentioned earlier.
-L129-130: Remind us here that the trip from The Netherlands to Milan is the example from the KH text you use - I had forgotten it.
-Fig. 1: Could you explain the format of the upper panel in the text and/or in the figure caption so it is easier to understand and interpret this and future figures? It took me a little while to understand at first as I am not used to this type of graphic, with the zero in the middle splitting positive and negative responses.
-L151: Could you give a brief description of what the Cronbach-Alpha score measures and its range, so we can interpret the alpha value?
-L154: You introduced the concept of ”effect” in the introduction but I think it would be useful to write it out as ”change in attitude” or something similar that is a bit more descriptive so we easily understand. Same for the title of section 3.1, which could be a bit more descriptive.
-Fig. 2: Add the corresponding values for each Likert scale statement in the legend so we know what a score of 4/5 means, and perhaps clarify in the text. I assume 4 corresponds to ”agree”, but it would be good to be sure.
-Fig. 3: Please remind us in the caption that SCT above/equal or below 4 refer to participants with high or low SCT respectively. Same comment for Fig. 4-8.
-L195-199: I’m curious, did you also look at the effect of prior likelihood to travel by train vs. plane on changes in the participants’ opinions on flying or train travel?
-Fig. 5: I think you could remove 3 out of the 4 legends as they are all the same.
-L213: Please specify that the results separated by likelihood to take the plane are not shown.
-L250-251: This should be mentioned in the results section as well. Same for L271-272.
-L254-255: It could also be an additional explanation, and not necessarily an alternative.
-L259-260: It did change the opinions on flying vs. train travel of some of the participants. Please rephrase or mention the subtlety as per general comment 6.
-L275: Could you remind us here what RQ2 is, as you do for RQ1 above.
-L279-280: This is also what you showed with Fig. 2.
-L299-302: Please mention the figure in the methods section as well.
-L308-309: This is not really what you show though. Please rephrase.
Technical corrections:
-L39-41: This sentence needs rephrasing.
-L52: I would personally prefer using words like "climate mitigation and adaption" instead of ”the fight against climate change”, but I understand that this is a personal choice.
-L182: Remove ”were” or add ”that” before it.
-L212: ”significantly differently”. Please check for other instances in the manuscript.
-L212-213: ”very” is missing in front of ”intelligent” and ”trustworthy”.
-L234: Should ”strongly” be ”fully”, as per the Fig. 8 legend?
-L271-272: Rephrase to ”most participants (>60% for all three conditions) found the text interesting, felt calm, […], and annoyed”. Otherwise it’s not clear that the >60% applies to all these emotions.
-L277: Missing ”of” before ”sensitivity”.
-L279: Missing ”a” before ”reflection”.
-L281-282: Missing ”perceived” before ”more personal”.
Citation: https://doi.org/10.5194/egusphere-2024-543-RC2 - AC2: 'Reply on RC2', Erik van Sebille, 27 Jun 2024
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2024-543', Sam Illingworth, 20 May 2024
Overview
The manuscript presents an interesting and relevant subject that is a good fit for Geoscience Communication. The study explores the effects of personalization in climate communication, specifically looking at the credibility of climate scientists. Overall, the manuscript is well-structured and easy to read. However, there are several areas that require further development to enhance the clarity and impact of the work.
Major Comments
- Introduction expansion. The introduction needs expanding, particularly around the IMPACTLAB instrument. It is currently unclear what this instrument is, how it is implemented, and its relevant strengths and weaknesses compared to other approaches. This additional detail will provide a stronger foundation for your study.
- Section 1.2 on personalization. While Section 1.2 on Personalization is good, it would benefit from a more thorough discussion of how other publics, such as lobbyists and politicians, might use this rhetorical technique to undermine the work of scientists. Additionally, exploring how scientists and science communicators can learn from this approach would be valuable. Including examples and references where this approach has worked effectively (or not) in contexts like vaccinations and anti-smoking campaigns would strengthen this section.
- Selection of KH article. Clarify how the KH article was chosen in terms of topic. Discuss the implications of this choice for the results. For instance, consider whether topics such as ocean acidification or sea level rises affecting Indigenous communities might have yielded different outcomes.
- Translation and personalization. The issue of translation requires further exploration. Discuss how personalization might differ in Dutch compared to English, and what this implies for other languages. This will help in understanding the broader applicability of your findings.
- SurveySwap details. Provide more information about SurveySwap. Explain what it is and how it works in terms of reliability. This will help readers assess the robustness of your methodology.
- Consent forms and ethical considerations. Expand on the discussion around consent forms. What risks and benefits did you consider and communicate to participants? How was the data stored securely? Include more details on these aspects. Additionally, bring some of the 'Ethical statement' (line 325) into the main body of the text, such as the classification of the study as low risk, which negated the need for further ethical review or privacy assessment.
- Determining personalization levels. Explain how you determined if a text was slightly or highly personalized. Acknowledge the potential subjectivity here and discuss how this assessment could be repeated reliably.
- Title scope. The study focuses on one example – sustainable travel. The current title, "The (non)effect of personalization in climate texts on the credibility of climate scientists," is too broad. Consider revising the title to better reflect the scope of the study, such as "The (non)effect of personalization in climate texts on the credibility of climate scientists: A case study on sustainable travel."
- The conclusions currently read more like a summary of the article. Expand this section to include the significance of the work and its implications for future studies. This will provide a more impactful closing to your manuscript.
Minor Comments
- Abstract. The abstract is well-crafted and easy to digest. No changes are necessary here.
- Figures and captions. The figures are clear, well-produced, and effectively support the text. However, the captions could benefit from a little more detail so they can be understood independently of the main body of the article.
- Section 2.5 on quantitative statistics. Section 2.5 on quantitative statistics for the control is sound and well-explained but does not integrate smoothly with the preceding and following sections. Improving the transitions will enhance the overall flow of the manuscript.
- Statistical tests explanation. Every time you introduce a statistical test or method, such as the Cronbach-Alpha score, provide an explanation of what the test is and what certain scores mean. This will help the reader understand the methodology better and make the manuscript more accessible.
- Discussion and Research Questions. The discussion would benefit from clearer presentation of the Research Questions and their integration throughout the text. This will help in drawing direct connections between your findings and the questions posed.
- Fair discussion of limitations. The discussion of limitations is very fair and well done. This transparency strengthens the credibility of your study.
Citation: https://doi.org/10.5194/egusphere-2024-543-RC1 -
AC1: 'Reply on RC1', Erik van Sebille, 28 May 2024
Dear Sam, tank you very much for these useful comments. In the attached pdf, we sketch how we will take these into account in a revised version of the manuscript. We will, however, wait until we also receive the comments of reviewer 2 before we upload that revision.
-
RC2: 'Comment on egusphere-2024-543', Louise Arnal, 14 Jun 2024
In this manuscript, the authors examine how personalizing a science text about travel modes and climate change influences participants' attitudes towards climate change and their perception of the writer's credibility. The study reveals that personalization had some effect on participants' emotions but did not significantly alter their attitudes or perceptions of the scientist's credibility. Overall, I found the manuscript very engaging and a thoughtful contribution to Geoscience Communication and the literature. However, adding more nuances to the results and discussion sections would enhance its depth. Below are my comments, which I hope will be helpful to revise your manuscript for publication.
General comments:
- It would be beneficial to expand the discussion on scientists expressing their opinions. While you introduce this concept on lines 58-70, revisiting it in the discussion with additional insights would be valuable. Often, scientists are guided by their institutions to avoid taking official stances. Given the urgency of the climate crisis, should scientists feel more comfortable advocating, and should we train future scientists to do this more effectively? Additionally, should workplaces support this type of communication, or should it be left to climate communication experts?
- Why was the personalization conducted in two steps instead of comparing a single personalized version to the original? This approach would have increased the sample sizes for each group. Please provide an explanation in the manuscript to clarify the rationale behind this methodology.
- Language use, including expressions and humour, is highly dependent on cultural context. A personalized text effective in one language would require careful translation to capture these nuances for different populations, whereas a more expository text might not need the same level of adjustment. Could you discuss this consideration in the manuscript?
- I couldn’t find the appendix.
- Much of the content in section 2.4 seems more appropriate for the results section. Please consider moving this material to a sub-section of the results.
- The results section could benefit from more detailed description in places. Firstly, some results indicate statistical significance between groups without specifying the direction of the effect. For example, on lines 211-213, we know that there is a difference in responses regarding the writer's perceived credibility, but it is unclear how prior likelihood to fly or science capital and trust (SCT) influence these responses. Similarly, this applies to lines 184-186 and 234-236 (please review for other instances). Secondly, even when results are not statistically significant, I think that it would still be useful to describe them qualitatively, clearly distinguishing between statistically significant and non-significant findings. For instance, regarding SCT on responses to control questions (Fig. 3), although not statistically significant, there is a qualitative difference where a higher percentage of participants with higher SCT found the texts professional/formal. Despite the sample size limiting the detection of small differences as you mention in the discussion, these can still be qualitatively appreciated. Adding such qualitative descriptions throughout the manuscript would highlight nuances not currently captured in the text but shown in the figures. Please consider discussing such subtleties, future research directions, and hypotheses for further studies with larger samples to explore these minimal effects more thoroughly.
- I find the question in section 3.3 about whether the text aims to persuade or inform intriguing, but it should be introduced earlier in the manuscript for better context. Additionally, the impact of participants’ SCT on their perception of the writer's impartiality (second question) is not discussed, warranting further exploration in the discussion section.
- Participants' prior knowledge (and possibly trust) of the KH platform and the writer's name could have influenced the results. Was this information provided to participants before they answered the questions? Clarifying this could help understand its impact.
Specific comments:
-L18: It would be useful to provide a brief description of the word ”attitude” in this context as it is very broad, perhaps using the text in between parentheses on L76.
-L20-22: This sentence talks about strengthening the link between climate information and action. There needs to be another sentence before that to discuss the existence of that link. Could you cite some literature that discusses the impact of (climate) information for changing attitudes and possibly actions?
-L25-31: I find that this more methodological piece would fit better later in the introduction, once the research questions have been introduced. Where it stands, it disrupts a bit the flow of a very interesting and broader introduction.
-L32: There is no section 1.1.
-L50-51: You already mentioned that attitudes affect climate action in the first sentences of the introduction. Perhaps rephrase this sentence slightly to incorporate it with the previous sentence and remove the repetition.
-L99-100: Could you include the full second sentence of the text so that it is a bit easier to interpret the meaning and changes made? Same for conditions 2 and 3.
-L116: Did all participants live in The Netherlands?
-L117-118: Why was the focus on young adults specifically? Please clarify this choice in the manuscript.
-L121: Does ”M” in the parentheses refer to mean or median? This could be written out for clarity and to match the format in section 2.4.1.
-L122: What are the results from the educational level? It is only mentioned in the discussion.
-L123: Will the questions be available in the appendix?
-L127: This is the first time that the word ”behaviour” is used in the manuscript. It would be useful to have a brief description of what you mean, and how it situates itself within the attitude, opinion, and action pieces you already mentioned earlier.
-L129-130: Remind us here that the trip from The Netherlands to Milan is the example from the KH text you use - I had forgotten it.
-Fig. 1: Could you explain the format of the upper panel in the text and/or in the figure caption so it is easier to understand and interpret this and future figures? It took me a little while to understand at first as I am not used to this type of graphic, with the zero in the middle splitting positive and negative responses.
-L151: Could you give a brief description of what the Cronbach-Alpha score measures and its range, so we can interpret the alpha value?
-L154: You introduced the concept of ”effect” in the introduction but I think it would be useful to write it out as ”change in attitude” or something similar that is a bit more descriptive so we easily understand. Same for the title of section 3.1, which could be a bit more descriptive.
-Fig. 2: Add the corresponding values for each Likert scale statement in the legend so we know what a score of 4/5 means, and perhaps clarify in the text. I assume 4 corresponds to ”agree”, but it would be good to be sure.
-Fig. 3: Please remind us in the caption that SCT above/equal or below 4 refer to participants with high or low SCT respectively. Same comment for Fig. 4-8.
-L195-199: I’m curious, did you also look at the effect of prior likelihood to travel by train vs. plane on changes in the participants’ opinions on flying or train travel?
-Fig. 5: I think you could remove 3 out of the 4 legends as they are all the same.
-L213: Please specify that the results separated by likelihood to take the plane are not shown.
-L250-251: This should be mentioned in the results section as well. Same for L271-272.
-L254-255: It could also be an additional explanation, and not necessarily an alternative.
-L259-260: It did change the opinions on flying vs. train travel of some of the participants. Please rephrase or mention the subtlety as per general comment 6.
-L275: Could you remind us here what RQ2 is, as you do for RQ1 above.
-L279-280: This is also what you showed with Fig. 2.
-L299-302: Please mention the figure in the methods section as well.
-L308-309: This is not really what you show though. Please rephrase.
Technical corrections:
-L39-41: This sentence needs rephrasing.
-L52: I would personally prefer using words like "climate mitigation and adaption" instead of ”the fight against climate change”, but I understand that this is a personal choice.
-L182: Remove ”were” or add ”that” before it.
-L212: ”significantly differently”. Please check for other instances in the manuscript.
-L212-213: ”very” is missing in front of ”intelligent” and ”trustworthy”.
-L234: Should ”strongly” be ”fully”, as per the Fig. 8 legend?
-L271-272: Rephrase to ”most participants (>60% for all three conditions) found the text interesting, felt calm, […], and annoyed”. Otherwise it’s not clear that the >60% applies to all these emotions.
-L277: Missing ”of” before ”sensitivity”.
-L279: Missing ”a” before ”reflection”.
-L281-282: Missing ”perceived” before ”more personal”.
Citation: https://doi.org/10.5194/egusphere-2024-543-RC2 - AC2: 'Reply on RC2', Erik van Sebille, 27 Jun 2024
Peer review completion
Journal article(s) based on this preprint
Data sets
Survey data Anna Leerink and Erik van Sebille https://github.com/erikvansebille/KH_personalization_effect/blob/main/KlimaatHelpdesk_20240105.csv
Interactive computing environment
Python notebook with analysis and plotting code Erik van Sebille https://github.com/erikvansebille/KH_personalization_effect/blob/main/analyse_KH_data.ipynb
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 266 | 81 | 33 | 380 | 12 | 11 |
- HTML: 266
- PDF: 81
- XML: 33
- Total: 380
- BibTeX: 12
- EndNote: 11
Viewed (geographical distribution)
| Country | # | Views | % |
|---|---|---|---|
| United States of America | 1 | 140 | 34 |
| Netherlands | 2 | 75 | 18 |
| United Kingdom | 3 | 27 | 6 |
| China | 4 | 25 | 6 |
| Germany | 5 | 21 | 5 |
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
- 140
Anna Leerink
Daan Reijnders
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(1157 KB) - Metadata XML