the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Understanding representations of uncertainty, an eye-tracking study part I: The effect of anchoring
Kelsey J. Mulder
Louis Williams
Matthew Lickiss
Alison Black
Andrew Charlton-Perez
Rachel McCloy
Eugene McSorley
Abstract. Geoscience communicators must think carefully about how uncertainty is represented and how users may interpret these representations. Doing so will help communicate risk more effectively, which can elicit appropriate responses. Recently, communication of uncertainty has come to the forefront over the course of the COVID-19 pandemic, but the lessons learned from communication during the pandemic can be adopted across geosciences as well. To test interpretations of environmental forecasts with uncertainty, a decision task survey was administered to 65 participants who saw different hypothetical forecast representations common to presentations of environmental data and forecasts: deterministic, spaghetti plot with and without a median line, fan plot with and without a median line, and box plot with and without a median line. While participants completed the survey, their eye movements were monitored with eye-tracking software. Participants’ eye movements were anchored to the median line, not focusing on possible extreme values to the same extent as when no median line was present. Additionally, participants largely correctly interpreted extreme values from the spaghetti and fan plots, but misinterpreted extreme values from the box plot, perhaps because participants spent little time fixating on the key. These results suggest that anchoring lines, such as median lines, should only be used where users should be guided to particular values and where extreme values are not as important in data interpretation. Additionally, fan or spaghetti plots should be considered instead of box plots to reduce misinterpretation of extreme values. Further study on the role of expertise and the change in eye movements across the graph area and key is explored in more detail in the companion paper to this study.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(8031 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(8031 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Kelsey J. Mulder et al.
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2022-927', Anonymous Referee #1, 17 Nov 2022
This is an important issue - not just for users, but for professionals formulating warning messages. Anchoring can be positive, e.g. if the anchor line is set at an agreed response threshold e.g. reasonable worst case, or for a specific cost/loss ratio, but it can also be negative especially in a low probability high impact situation, if highlighting of the median leads to underestimation of risk.
Line 84: It should be noted that the sample is not representative of typical users. I don't believe this undermines ther esults in any way, but a comment on the differnces between the sample and typical real life users might be worth including.
Line 105-6: The meaning of the three probability levels is not clear. I believe it may mean that there were three forecast scenarios presented with data adjusted to give 30%, 50% and 70% probabilities of exceeding 1 metre, and that each was presented in 7 different ways. However, I am still not sure if that is a correct interpretation. There is no figure that shows what the three scenarios look like (maybe the deterministic presentations would clearly display them?). I think it would also be helpful to include a brief description of how the 3 scenarios were created.
Line 131: The authors should not assume that all readers will be familiar with the fit statistics for the ANOVA or Bonferroni tests. I suggest adding a short section 2.3 to introduce the statistical tests used and the meaning of the fit statistics (supplemented by a suitable reference) .
Line 179: The explanation of the reason that goodinterpretations were made of the spaghetti plots despite even less attention to the key than for the boxplots may be correct, but it runs counter to the general view that spaghetti plots should be avoided because they are difficult to interpret (due to crossing lines etc). I think this view should be acknowledged and responded to.
Line 189: This paragraph does not tell us what the impact on answer time actually is, merely what the statistics are of the differences. We should not have to look at the figure to work this out.
Line 198: a comment on the increase in time to estimate the maximum in a spaghetti plot with median is needed.
Line 211: This line starts with "However", which seems out of place given that the previous paragraph was also talking about the dangers of anchoring.
Citation: https://doi.org/10.5194/egusphere-2022-927-RC1 -
AC1: 'Reply on RC1', Eugene McSorley, 30 Jan 2023
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2022/egusphere-2022-927/egusphere-2022-927-AC1-supplement.pdf
-
AC1: 'Reply on RC1', Eugene McSorley, 30 Jan 2023
-
RC2: 'Comment on egusphere-2022-927', Anonymous Referee #2, 20 Dec 2022
General Comments: I have major concerns with this manuscript as written. First and foremost, I find that there’s a mismatch between how the results are reported and what is described in the methods. The author’s stated focus in this article is on the results of an eye-tracking study of different uncertainty visualizations, but the authors also refer at times to a survey that was conducted along with the eye-tracking results. As it stands, however, the authors barely describe this survey in the methods and don’t provide any of the survey results in the paper proper (with a brief discussion tucked away in the appendix). These results are necessary to contextualize the eye-tracking results, and so the result is a paper that feels half done. There are other major issues to contend with as well – the authors frame this article in terms of COVID-19, when it is not clear how these uncertainty visualizations are related to COVID-19; there are some spots in the intro where the authors do not provide enough supporting literature; and there are organizational issues throughout the paper that make it difficult to follow. The eye-tracking results, depending on how they are framed, could be interesting, especially when paired with appropriate survey/decision task analysis, but as currently written, I believe the authors need to go back to the drawing board.
Major Comments:
Lines 3-4 and 17-19 – I’m confused by the reference to the COVID-19 pandemic. Are the visuals you study in this analysis commonly used to visualize COVID? Just seems like this framing doesn’t really fit, especially when the rest of the intro is focused on applications in geosciences
25-27 – feels like there should be a few more citations here, if there has truly been a “much greater volume of geoscience research” on these topics
30-31 – unclear what this means – implying that narrative consistency approach is incompatible with ensemble predictions?
50-60 - this paragraph feels a bit garbled…not sure what the key takeaway is here. One thing I might suggest is devoting an entire paragraph to the previous research from your research group – splitting it between two paragraphs breaks up the flow. And this might make it easier to establish what your previous research has already established, and what this study is trying to learn to build on those results.
56 – it might be helpful to explain/provide examples of the types of visuals you look at in more detail somewhere in the introduction. Fan plots, for instance, are not something I was familiar with before reading this.
70-71 – might cite Sutton & Fischer (2021) or Sutton et al (2020) here or elsewhere in the intro – not related specifically to your topic of study, but provides some previous examples of eye-tracking in geoscience communication
72-75 – some thoughts on these RQs. First, I think it would be useful to define the various types of data presentations you’re interested in looking at (e.g. box plot, fan plot, spaghetti plot with/without medians) before leading in here. At this stage, it’s unclear what is meant by “data presentation”. Second, the RQs are too vague and mismatched with your results. The first section of results focuses on how the presence of a median line affects where participants eyes go when making decisions, and so I might reframe this question to be more in line with how central tendency lines affect visual processing of risk visualizations. Likewise, the second set of results focuses on how participants engage with the information in the key of the visuals – I’m not sure how this is related to “uncertainty information”, except in a very broad sense. The third RQ is better, but I might be even more specific in asking whether participants take more time to make decisions based on visual attributes – it’s just hard to tell from these results whether the time taken to make a decision is due to cognitive load or some other factor. Also, I’m not sure this final RQ is really supported by the literature provided in the introduction. Why is this concept important?
84-87 – might lead off this paragraph with the description of your participants, then talk about measures
88-94 – this paragraph feels a bit misplaced – perhaps it should be included in the next section, when you talk about the decision tasks that were part of the survey? This paragraph is also unclear because you say that you used the same survey instrument as in a previous study, but also mention that the previous study had a different focus. Does this mean that you also asked about economic rationality of decisions in this study, and didn’t report the results from those questions? Did you also use eye-tracking in the previous study? Was the experimental method the same between the two studies, just with a different population? It sounds from the way you phrase this that the previous study did not use eye-tracking as part of the experimental method. If this is the case, then I’m not sure the replication information is useful, considering the varying conditions in which the participants were answering the question.
95 – do you make it clear in this section that participants are asked to make a decision based on a 72-hour forecast? Why this forecast lead-time?
111 – I’m curious why you mention the minimum possible decision task, as it doesn’t seem that you report on it here. More broadly, I think you can make the decision tasks more clear, and maybe explain why you chose those decision tasks (e.g., what are they operationalizing? Why are these tasks important? What do they elucidate?). Details about how the decision task survey was administered would be useful as well.
112 – I think it would be useful to define your dependent variables in this section – e.g. you talk in the results about the duration of fixation, the time taken to look at various elements, etc., and it might be helpful to lay these out and explain what they represent and why they’re important here
121 – You lead off the results by noting that eye-tracking helps to explain anchoring seen in the survey results, but you haven’t presented any survey results. What results are you talking about then? Are these results from a previous survey? The results presented in the appendix? A forthcoming publication? As you note here, these survey results are essential to understand and interpret the eye-tracking results.
123-124 – there’s too much information loaded into the parentheses – maybe note somewhere earlier in this section that you only visualize results for the spaghetti plot and that other visuals are in supplementary information? But I guess this raises a more important point – why did you only choose to highlight the spaghetti plot visuals?
128 – another related concern – in this section, you talk exclusively about the ship decision and maximum ice thickness forecast decision tasks. Where are the results for the other decision tasks asked of participants?
128-130 – I don’t see ship decision in Figure 3? Unless ship decision just means best-guess forecast, but it seems like those were different tasks
138-142 – I would question how practical some of these results are. For instance, the median vs no median variance for spaghetti at 30% doesn’t appear to be significantly different
143 – this section header seems inappropriate – this section is mostly focused on legend fixations for the various data presentations, so I would be more specific. “Interpretations” is a bit too broad for me.
143 – general question for this section – are these results for one decision task or for all decisions aggregated? Would be helpful to know
144-151 – this discussion feels misplaced here, considering these are results from a previous study which should have been covered in the introduction. It gives the impression that these are results from this study, but this doesn't seem to be the case.
150-151 – clarify this sentence – are you saying that the maxima provided by participants was close to the highest member of the spaghetti plot?
152 – passive language. Also, based on what? This is a possible assertion, but it is not proven by any of the results that follow
152-153 – from the results that follow, it seems that you recorded the number of seconds fixating on the key and the number of fixations on the key for all image types, not just for the first box plot. So this sentence needs to be clarified.
162-163 – clarify here – do you mean that as the participants were exposed to additional box plots as part of the experimental design, they fixated less on the key? This raises additional questions about how the results in this section are reported – do the mean fixations and time spent fixating on the key include every time the participants viewed the specified visualizations, or just the first time? Also, do you see a similar pattern of fewer fixations/time spent fixating on the key for the other visualization types, or just box plot?
168-181 – these last three paragraphs again reference the “survey” results which are not reported elsewhere and which are needed to make sense of the eye-tracking results
170-171 – sentence beginning “It took longer before…” – isn’t this just rehashing what is stated in the first two sentences of this paragraph?
183-185 – great lead into this section, but is this question really asked anywhere in the intro (outside of the research questions)?
187-188 – what is meant by “first two series of questions”? Were any of the probabilities or visualization conditions overrepresented among these first two series? Also, do you take this adjustment period into account for the other analyses reported in earlier sections?
189 – unclear on first read that “anchoring” here refers to including the median line – might be better to be specific that including the median line affected the amount of time needed to make a decision
198 – the previous sections had some level of discussion intertwined with the results – here, that’s missing and it’s really needed. What do these results mean practically? How should they be interpreted?
191-192 – interacting effects of “anchoring” and probability level? Would be helpful to explain what this interaction means practically
193-194 – the results reported here (significant main effect of anchoring on maximum forecast) is another example where the results may be significant but do not appear practically significant on the plots
215-225 – again, you mention survey responses despite not providing the survey results in this paper. And as is clear here, these results are essential to making any sort of conclusions from this data.
226 – “first”? Feels like some transition to “recommendations” is missing here
227 – curious about average seasonal temperatures as an example here, and can’t help but feel that a better example could have elucidated the concept more clearly. For instance, you could note that these median lines might be helpful in hurricane track forecasts where confidence in the track is high.
234-235 – again, curious about the COVID framing here
237-238 – these last two sentences feel like an odd way to end the paper. I think these points might be better included elsewhere, and then close the paper with the key takeaway message.
240 – maybe this is where you talk about the survey responses? But nonetheless, my point stands because these results need to be in the paper proper, and not tucked away in an appendix. I would pull out the survey responses from this study and place them in the actual paper, and then maybe you could leave the comparison between studies in the appendix
Figure 2 – I’m wondering if the information in the caption about centering should be included somewhere in the methods.
Figure 6 – really feel that these figures should be in the main paper – the patterns of fixation helpfully illustrate the patterns explored in section 3.2, in many cases more so than the box plots provided in Figure 3
Figure 7 – is there any explanation for the difference in fixation on the key for the median fan plot but not for the no median fan plot? Is this an ordering effect?
Minor Comments/Grammar/Typos:
20-22 – grammar issue - there’s a noun missing before “runs”
29 – misplaced question mark in citation?
59 – awkward phrasing…remove one of the “can”s
62 – “context and prior experience and the inherent limitations” – rephase to add commas
66-68 – add some sort of punctuation between the annotation and the references in parentheses (e.g., detecting a lesion in a mammogram; Kundel et al., 2007)
96-98 – first two sentences of this paragraph are worded awkwardly; revise
107 – “were” should be “was”
150 – the information in the parentheses should be added to the end of the sentence
177-180 – “reason for the little attention paid” – first two sentences of this paragraph use passive language; revise
178 – semi-colon needed before “however”
191 – “to make ship decision” – revise
228-229 – awkward phrasing with all the comma splices
241 – Mulder et al 2020 and 2020 – is this referring to two papers?
Figure 1 – the text in the graphics (especially the axis ticks, labels, and annotations) is quite small and hard to read…would encourage making it larger
Figure 2 – “between each question, there a cross was present” – grammar, fix
Figure 2 – “centring” should be “centering”
Figure 3 – there’s a LOT going on here – I think some of the smaller features (e.g. the stars as means, the blue squares and lines) are really difficult to see
Figure 5 – the annotations in the caption (e.g., b) confidence in decision) don’t line up with the annotations in the plot (where d is the confidence plot)
Citation: https://doi.org/10.5194/egusphere-2022-927-RC2 -
AC2: 'Reply on RC2', Eugene McSorley, 30 Jan 2023
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2022/egusphere-2022-927/egusphere-2022-927-AC2-supplement.pdf
-
AC2: 'Reply on RC2', Eugene McSorley, 30 Jan 2023
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2022-927', Anonymous Referee #1, 17 Nov 2022
This is an important issue - not just for users, but for professionals formulating warning messages. Anchoring can be positive, e.g. if the anchor line is set at an agreed response threshold e.g. reasonable worst case, or for a specific cost/loss ratio, but it can also be negative especially in a low probability high impact situation, if highlighting of the median leads to underestimation of risk.
Line 84: It should be noted that the sample is not representative of typical users. I don't believe this undermines ther esults in any way, but a comment on the differnces between the sample and typical real life users might be worth including.
Line 105-6: The meaning of the three probability levels is not clear. I believe it may mean that there were three forecast scenarios presented with data adjusted to give 30%, 50% and 70% probabilities of exceeding 1 metre, and that each was presented in 7 different ways. However, I am still not sure if that is a correct interpretation. There is no figure that shows what the three scenarios look like (maybe the deterministic presentations would clearly display them?). I think it would also be helpful to include a brief description of how the 3 scenarios were created.
Line 131: The authors should not assume that all readers will be familiar with the fit statistics for the ANOVA or Bonferroni tests. I suggest adding a short section 2.3 to introduce the statistical tests used and the meaning of the fit statistics (supplemented by a suitable reference) .
Line 179: The explanation of the reason that goodinterpretations were made of the spaghetti plots despite even less attention to the key than for the boxplots may be correct, but it runs counter to the general view that spaghetti plots should be avoided because they are difficult to interpret (due to crossing lines etc). I think this view should be acknowledged and responded to.
Line 189: This paragraph does not tell us what the impact on answer time actually is, merely what the statistics are of the differences. We should not have to look at the figure to work this out.
Line 198: a comment on the increase in time to estimate the maximum in a spaghetti plot with median is needed.
Line 211: This line starts with "However", which seems out of place given that the previous paragraph was also talking about the dangers of anchoring.
Citation: https://doi.org/10.5194/egusphere-2022-927-RC1 -
AC1: 'Reply on RC1', Eugene McSorley, 30 Jan 2023
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2022/egusphere-2022-927/egusphere-2022-927-AC1-supplement.pdf
-
AC1: 'Reply on RC1', Eugene McSorley, 30 Jan 2023
-
RC2: 'Comment on egusphere-2022-927', Anonymous Referee #2, 20 Dec 2022
General Comments: I have major concerns with this manuscript as written. First and foremost, I find that there’s a mismatch between how the results are reported and what is described in the methods. The author’s stated focus in this article is on the results of an eye-tracking study of different uncertainty visualizations, but the authors also refer at times to a survey that was conducted along with the eye-tracking results. As it stands, however, the authors barely describe this survey in the methods and don’t provide any of the survey results in the paper proper (with a brief discussion tucked away in the appendix). These results are necessary to contextualize the eye-tracking results, and so the result is a paper that feels half done. There are other major issues to contend with as well – the authors frame this article in terms of COVID-19, when it is not clear how these uncertainty visualizations are related to COVID-19; there are some spots in the intro where the authors do not provide enough supporting literature; and there are organizational issues throughout the paper that make it difficult to follow. The eye-tracking results, depending on how they are framed, could be interesting, especially when paired with appropriate survey/decision task analysis, but as currently written, I believe the authors need to go back to the drawing board.
Major Comments:
Lines 3-4 and 17-19 – I’m confused by the reference to the COVID-19 pandemic. Are the visuals you study in this analysis commonly used to visualize COVID? Just seems like this framing doesn’t really fit, especially when the rest of the intro is focused on applications in geosciences
25-27 – feels like there should be a few more citations here, if there has truly been a “much greater volume of geoscience research” on these topics
30-31 – unclear what this means – implying that narrative consistency approach is incompatible with ensemble predictions?
50-60 - this paragraph feels a bit garbled…not sure what the key takeaway is here. One thing I might suggest is devoting an entire paragraph to the previous research from your research group – splitting it between two paragraphs breaks up the flow. And this might make it easier to establish what your previous research has already established, and what this study is trying to learn to build on those results.
56 – it might be helpful to explain/provide examples of the types of visuals you look at in more detail somewhere in the introduction. Fan plots, for instance, are not something I was familiar with before reading this.
70-71 – might cite Sutton & Fischer (2021) or Sutton et al (2020) here or elsewhere in the intro – not related specifically to your topic of study, but provides some previous examples of eye-tracking in geoscience communication
72-75 – some thoughts on these RQs. First, I think it would be useful to define the various types of data presentations you’re interested in looking at (e.g. box plot, fan plot, spaghetti plot with/without medians) before leading in here. At this stage, it’s unclear what is meant by “data presentation”. Second, the RQs are too vague and mismatched with your results. The first section of results focuses on how the presence of a median line affects where participants eyes go when making decisions, and so I might reframe this question to be more in line with how central tendency lines affect visual processing of risk visualizations. Likewise, the second set of results focuses on how participants engage with the information in the key of the visuals – I’m not sure how this is related to “uncertainty information”, except in a very broad sense. The third RQ is better, but I might be even more specific in asking whether participants take more time to make decisions based on visual attributes – it’s just hard to tell from these results whether the time taken to make a decision is due to cognitive load or some other factor. Also, I’m not sure this final RQ is really supported by the literature provided in the introduction. Why is this concept important?
84-87 – might lead off this paragraph with the description of your participants, then talk about measures
88-94 – this paragraph feels a bit misplaced – perhaps it should be included in the next section, when you talk about the decision tasks that were part of the survey? This paragraph is also unclear because you say that you used the same survey instrument as in a previous study, but also mention that the previous study had a different focus. Does this mean that you also asked about economic rationality of decisions in this study, and didn’t report the results from those questions? Did you also use eye-tracking in the previous study? Was the experimental method the same between the two studies, just with a different population? It sounds from the way you phrase this that the previous study did not use eye-tracking as part of the experimental method. If this is the case, then I’m not sure the replication information is useful, considering the varying conditions in which the participants were answering the question.
95 – do you make it clear in this section that participants are asked to make a decision based on a 72-hour forecast? Why this forecast lead-time?
111 – I’m curious why you mention the minimum possible decision task, as it doesn’t seem that you report on it here. More broadly, I think you can make the decision tasks more clear, and maybe explain why you chose those decision tasks (e.g., what are they operationalizing? Why are these tasks important? What do they elucidate?). Details about how the decision task survey was administered would be useful as well.
112 – I think it would be useful to define your dependent variables in this section – e.g. you talk in the results about the duration of fixation, the time taken to look at various elements, etc., and it might be helpful to lay these out and explain what they represent and why they’re important here
121 – You lead off the results by noting that eye-tracking helps to explain anchoring seen in the survey results, but you haven’t presented any survey results. What results are you talking about then? Are these results from a previous survey? The results presented in the appendix? A forthcoming publication? As you note here, these survey results are essential to understand and interpret the eye-tracking results.
123-124 – there’s too much information loaded into the parentheses – maybe note somewhere earlier in this section that you only visualize results for the spaghetti plot and that other visuals are in supplementary information? But I guess this raises a more important point – why did you only choose to highlight the spaghetti plot visuals?
128 – another related concern – in this section, you talk exclusively about the ship decision and maximum ice thickness forecast decision tasks. Where are the results for the other decision tasks asked of participants?
128-130 – I don’t see ship decision in Figure 3? Unless ship decision just means best-guess forecast, but it seems like those were different tasks
138-142 – I would question how practical some of these results are. For instance, the median vs no median variance for spaghetti at 30% doesn’t appear to be significantly different
143 – this section header seems inappropriate – this section is mostly focused on legend fixations for the various data presentations, so I would be more specific. “Interpretations” is a bit too broad for me.
143 – general question for this section – are these results for one decision task or for all decisions aggregated? Would be helpful to know
144-151 – this discussion feels misplaced here, considering these are results from a previous study which should have been covered in the introduction. It gives the impression that these are results from this study, but this doesn't seem to be the case.
150-151 – clarify this sentence – are you saying that the maxima provided by participants was close to the highest member of the spaghetti plot?
152 – passive language. Also, based on what? This is a possible assertion, but it is not proven by any of the results that follow
152-153 – from the results that follow, it seems that you recorded the number of seconds fixating on the key and the number of fixations on the key for all image types, not just for the first box plot. So this sentence needs to be clarified.
162-163 – clarify here – do you mean that as the participants were exposed to additional box plots as part of the experimental design, they fixated less on the key? This raises additional questions about how the results in this section are reported – do the mean fixations and time spent fixating on the key include every time the participants viewed the specified visualizations, or just the first time? Also, do you see a similar pattern of fewer fixations/time spent fixating on the key for the other visualization types, or just box plot?
168-181 – these last three paragraphs again reference the “survey” results which are not reported elsewhere and which are needed to make sense of the eye-tracking results
170-171 – sentence beginning “It took longer before…” – isn’t this just rehashing what is stated in the first two sentences of this paragraph?
183-185 – great lead into this section, but is this question really asked anywhere in the intro (outside of the research questions)?
187-188 – what is meant by “first two series of questions”? Were any of the probabilities or visualization conditions overrepresented among these first two series? Also, do you take this adjustment period into account for the other analyses reported in earlier sections?
189 – unclear on first read that “anchoring” here refers to including the median line – might be better to be specific that including the median line affected the amount of time needed to make a decision
198 – the previous sections had some level of discussion intertwined with the results – here, that’s missing and it’s really needed. What do these results mean practically? How should they be interpreted?
191-192 – interacting effects of “anchoring” and probability level? Would be helpful to explain what this interaction means practically
193-194 – the results reported here (significant main effect of anchoring on maximum forecast) is another example where the results may be significant but do not appear practically significant on the plots
215-225 – again, you mention survey responses despite not providing the survey results in this paper. And as is clear here, these results are essential to making any sort of conclusions from this data.
226 – “first”? Feels like some transition to “recommendations” is missing here
227 – curious about average seasonal temperatures as an example here, and can’t help but feel that a better example could have elucidated the concept more clearly. For instance, you could note that these median lines might be helpful in hurricane track forecasts where confidence in the track is high.
234-235 – again, curious about the COVID framing here
237-238 – these last two sentences feel like an odd way to end the paper. I think these points might be better included elsewhere, and then close the paper with the key takeaway message.
240 – maybe this is where you talk about the survey responses? But nonetheless, my point stands because these results need to be in the paper proper, and not tucked away in an appendix. I would pull out the survey responses from this study and place them in the actual paper, and then maybe you could leave the comparison between studies in the appendix
Figure 2 – I’m wondering if the information in the caption about centering should be included somewhere in the methods.
Figure 6 – really feel that these figures should be in the main paper – the patterns of fixation helpfully illustrate the patterns explored in section 3.2, in many cases more so than the box plots provided in Figure 3
Figure 7 – is there any explanation for the difference in fixation on the key for the median fan plot but not for the no median fan plot? Is this an ordering effect?
Minor Comments/Grammar/Typos:
20-22 – grammar issue - there’s a noun missing before “runs”
29 – misplaced question mark in citation?
59 – awkward phrasing…remove one of the “can”s
62 – “context and prior experience and the inherent limitations” – rephase to add commas
66-68 – add some sort of punctuation between the annotation and the references in parentheses (e.g., detecting a lesion in a mammogram; Kundel et al., 2007)
96-98 – first two sentences of this paragraph are worded awkwardly; revise
107 – “were” should be “was”
150 – the information in the parentheses should be added to the end of the sentence
177-180 – “reason for the little attention paid” – first two sentences of this paragraph use passive language; revise
178 – semi-colon needed before “however”
191 – “to make ship decision” – revise
228-229 – awkward phrasing with all the comma splices
241 – Mulder et al 2020 and 2020 – is this referring to two papers?
Figure 1 – the text in the graphics (especially the axis ticks, labels, and annotations) is quite small and hard to read…would encourage making it larger
Figure 2 – “between each question, there a cross was present” – grammar, fix
Figure 2 – “centring” should be “centering”
Figure 3 – there’s a LOT going on here – I think some of the smaller features (e.g. the stars as means, the blue squares and lines) are really difficult to see
Figure 5 – the annotations in the caption (e.g., b) confidence in decision) don’t line up with the annotations in the plot (where d is the confidence plot)
Citation: https://doi.org/10.5194/egusphere-2022-927-RC2 -
AC2: 'Reply on RC2', Eugene McSorley, 30 Jan 2023
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2022/egusphere-2022-927/egusphere-2022-927-AC2-supplement.pdf
-
AC2: 'Reply on RC2', Eugene McSorley, 30 Jan 2023
Peer review completion
Journal article(s) based on this preprint
Kelsey J. Mulder et al.
Kelsey J. Mulder et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 302 | 132 | 17 | 451 | 5 | 5 |
- HTML: 302
- PDF: 132
- XML: 17
- Total: 451
- BibTeX: 5
- EndNote: 5
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(8031 KB) - Metadata XML