the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 27 Aug 2025
Quantifying the Impact of Skeptical Science Rebuttals in Reducing Climate Misperceptions
Abstract. Misinformation about climate change causes societal damage in a number of ways and consequently, resources are required to support interventions that counter their influence. Aiming to meet this need, Skeptical Science is a highly-visited website featuring 250 rebuttals of misinformation about climate change. The rebuttals are written at multiple levels – basic, intermediate, and advanced – in order to reach as wide an audience as possible. This study collects survey data from visitors to the website, measuring their belief in climate facts and myths before and after reading a rebuttal. Our data found that a plurality of visitors were already highly convinced regarding climate facts, indicating many visitors come to the site not to answer unresolved questions but to gather resources and answers. We found that the rebuttals were effective in reducing belief in climate myths but that some rebuttals show a concerning reduction in belief in climate facts. The greatest improvement occurred with visitors who began with the most inaccurate climate perceptions. This indicates that the website is useful for two main audiences – those who are convinced about climate change but looking for material to support their own climate communication efforts, and those who disagree with climate facts but are open to new information. We examine potential ways that Skeptical Science rebuttals could be updated to improve their performance in raising climate literacy and critical thinking skills.
- Preprint
(2139 KB) - Metadata XML
- BibTeX
- EndNote
Status: open (until 22 Oct 2025)
-
RC1: 'Comment on egusphere-2025-3812', Anonymous Referee #1, 31 Aug 2025
reply
-
CC1: 'Reply on RC1', David Crookall, 05 Sep 2025
reply
This is David Crookall, lead guest ed of the special issue.
Just a quick note to say thank you for an extremely helpful review.
Citation: https://doi.org/10.5194/egusphere-2025-3812-CC1 -
AC1: 'Reply on CC1', John Cook, 15 Sep 2025
reply
Likewise, on behalf of the coauthors of the paper, thanks for your very helpful comments - we look forward to implementing them and strengthening our paper.
Citation: https://doi.org/10.5194/egusphere-2025-3812-AC1
-
AC1: 'Reply on CC1', John Cook, 15 Sep 2025
reply
-
CC1: 'Reply on RC1', David Crookall, 05 Sep 2025
reply
-
RC2: 'Comment on egusphere-2025-3812', Anonymous Referee #2, 10 Sep 2025
reply
This article presents the results of a survey on the website Skeptical Science and attempts to determine the degree to which the climate myth rebuttals on the website changes the visitors’ viewpoint on the scientific consensus about climate change. This work is evidently very important in today’s world, where climate misinformation is becoming easier to spread while the need to act on the changing climate is all the more urgent. Therefore, I believe that the research in this piece of work to be highly publishable in this journal.
However, I recommend that the authors improve the quality of some aspects of the article to make it easier to follow, and to avoid making assumptions about why some rebuttals seemingly did not perform well, without any qualitative evidence to back this up (see below).
Introduction – It feels like there is possibly too much background information in this section, especially as it is presented without context as to why it is necessary to know before reading the rest of the study. In particular, I do not feel like Section 1.2 (Evolution of SkS Rebuttals) is necessary as it has no relevance to the survey or the discussion of the study. I acknowledge that background information about SkS is essential, but feel like Section 1.1 alone is adequate in preparing the reader for the rest of the article.A glaring omission when reading the introduction was a specific research aim, or purpose of the study. All the background information was presented without knowing the context behind it – why do we need to know what has been described in Section 1-1.1? What hypothesis or research question are you seeking to answer by doing this study?
Methods – I feel like a lot of trust is being placed on the assumption that all visitors that completed the second survey read the webpage and took in what was being said. For example, some people could have quickly scrolled to the bottom of the webpage and filled in the survey without reading the content of the website. Has some filter been applied where responses completed below a minimum time are removed from analysis? If not, I feel this should be considered. On a related note, I can’t seem to find Table S1 in Supplementary Material (where there is supposedly a distribution of times) – only Appendices.
Results – It would be interesting to know whether the “level” of the rebuttal read by the visitors had any affect on their change in perception of climate facts (though I concede this is probably impossible to find out in retrospect). However, my main concern with this section comes from the analysis of individual rebuttals (from Fig. 5) and why the “worst-performing” ones are not believed. Most of the arguments here as to how the rebuttals could be improved come from the authors, and are not the result of the survey itself. Therefore, I find it confusing to make comments like “rebuttal x fails to take in to account y and z” when there is no qualitative evidence from the survey that this is why visitors did not believe the rebuttal. While I think the comments made about these individual rebuttals in this way are useful and the reasoning is sound, this more speculative analysis would be better placed in the discussion section.
Discussion – I take issue with the statement on line 182 that suggests that 45% is “most visitors” – it is less than half! I do however agree that more visitors to the site agreed with climate facts than didn’t, so this sentence should be rephrased for clarity. I do not, however, agree that the implication (line 184) of this means that most visitors are “looking for information to assist them in responding to climate misinformation” – again this is not asked in the survey. These visitors could have been looking for detailed, accurate scientific details about certain aspects of climate change from a reliable source for individual research purposes, or just general interest. The wording around this implication could therefore be loosened to accept that other reasons exist as to why such people would visit the site.
Citation: https://doi.org/10.5194/egusphere-2025-3812-RC2 -
CC2: 'Thank you', David Crookall, 14 Sep 2025
reply
This is David Crookall, lead guest ed of the special issue.
Just a quick note to say thank you for an extremely helpful review.
Citation: https://doi.org/10.5194/egusphere-2025-3812-CC2 -
AC2: 'Reply on CC2', John Cook, 15 Sep 2025
reply
Thanks for your very helpful comments. You make a robust point - 45% is not "most"! :-) We look forward to implementing your comments in our revision which will improve our paper.
Citation: https://doi.org/10.5194/egusphere-2025-3812-AC2
-
AC2: 'Reply on CC2', John Cook, 15 Sep 2025
reply
-
CC2: 'Thank you', David Crookall, 14 Sep 2025
reply
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 342 | 27 | 7 | 376 | 13 | 10 |
- HTML: 342
- PDF: 27
- XML: 7
- Total: 376
- BibTeX: 13
- EndNote: 10
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
This manuscript reports the findings of an in vivo impact evaluation of climate myth debunkings on people who arrived at Skeptical Science via a Google search query. Assessing the impact of attempts to debunk climate misinformation in vivo is a worthy research goal, and the research reported in this manuscript appears to be well conducted. Overall, I find this paper to be useful and will be pleased to see it published.
That said, I encourage the authors to consider how they might make the paper easier to read and grasp their key learning.
The Introduction section, for example, is a bit disjointed. Lines 26 to 42 could be restructured to clarify the more “subtle and subversive impacts” of climate misinformation. It isn’t clear why the “evolution of SKS rebuttals” section was included, given that the focus of the paper is an evaluation of current SKS content. Moreover, as a reader, I would like to see additional citations to substantiate statements about relevant prior research.
In the Methods, I would like to see a better explanation of how participants arrived at the research experience. The paper states they arrived via a Google search, but it would help readers understand the situation better if we knew what they were looking for. Is it the case that the Google research took them directly to a specific debunking and we safely presume that they asked Google a direct question related to that debunking? Readers need to better understand the research participant’s information search expectations to understand how they may have reacted to the question asked by the researchers. A minor point: I find the term “modal” (as in “modal pop-up screen”) confusing in this context. Is there an alternative term that can be used instead?
The results section is currently a bit hard to follow. The current flow of information is, essentially, “here is the data” and “this is what that data means.” I realize this is a standard and common way of reporting research findings, but I encourage the authors consider inverting the order of presentation to make easier for readers understand what the data mean. In essence, I’m suggesting the authors lead with a series of statements about the finding (e.g., the debunkings made visitors less likely to believe the climate myths being debunked) and then show the data that supports each claim.
I found the qualitative investigation of the best and worst performing rebuttals to be extremely useful, but I also it difficult to follow. Here too, I feel the detailed explanations made it challenging for readers to grasp the key learnings. Here too, I recommend leading with the key learning and then providing the details that justify the conclusion.
Lastly, in the Discussion section, I was not clear on why the authors concluded that “most visitors (45%) already strongly agreed with climate facts” (because 45% is not “most”), or that “they were looking for information to assist them in responding to climate misinformation.” I’m not saying these conclusions are unjustified, but I am saying I would find the statements more credible if they were better defended.
The paragraph beginning on line 201, which addresses why some of the debunkings performed poorly, would be better placed after the discussion of the overall pattern of findings—that belief in climate myths declined, but so too did belief in climate facts.
Minor points:
Line 26: the phrase “climate perceptions” does not fully convey the intended idea.
Line 224 : The phrase “a decreas in climate facts” does not accurately convey the intended idea.
Conclusion:
This paper is publication-worthy, but I would prefer to see it strengthened before publication. I thank the authors for their important work.