the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 05 Oct 2023
A comparison of two causal methods in the context of climate analyses
Abstract. Correlation does not necessarily imply causation, and this is why causal methods have been developed to try to disentangle true causal links from spurious relationships. In our study, we use two causal methods, namely the Liang-Kleeman information flow (LKIF) and the Peter and Clark momentary conditional independence (PCMCI) algorithm, and apply them to four different artificial models of increasing complexity and one real-case study based on climate indices in the North Atlantic and North Pacific. We show that both methods are superior to the classical correlation analysis, especially in removing spurious links. LKIF and PCMCI display some strengths and weaknesses for the three simplest models, with LKIF performing better with a smaller number of variables, and PCMCI being best with a larger number of variables. Detecting causal links from the fourth model is more challenging as the system is nonlinear and chaotic. For the real-case study with climate indices, both methods present some similarities and differences at monthly time scale. One of the key differences is that LKIF identifies the Arctic Oscillation (AO) as the largest driver, while El Niño-Southern Oscillation (ENSO) is the main influencing variable for PCMCI. More research is needed to confirm these links, in particular including nonlinear causal methods.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(2072 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2072 KB) - Metadata XML
- BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2023-2212', Anonymous Referee #1, 13 Nov 2023
The authors apply two causal inference methods to a variety of test models as well as a climate example. The paper is interesting and worthy of publication pending some revisions for clarity.
Comments:
– L25–7: Although instantaneous correlations cannot determine the causal direction, lagged regressions are often used for this purpose, despite their drawbacks compared to causal inference methods. It may be worth citing McGraw and Barnes (2018) here, since they support your point about the need for causality analysis rather than lagged regressions for climate studies.
– L92: The notation dw_k ∼ sqrt(dt) N(0, 1) is unusual. Can you please use the typical notation in which this is stated using non-infinitesimal increments, i.e., something like w_k(t+t_0) - w_k(t_0) ∼ sqrt(t) N(0, 1)?
– Eq. 5: In Liang and Kleeman (2005), they derive a different expression for discrete and continuous time. Do you need to use different expressions for LKIF here in either case?
– L211–2: There is not enough information about how the significance was computed. What is the null hypothesis, and how exactly was the bootstrap distribution used? Please add details. Also, the significance test for PCMCI is not described.
– L222–4: The conditions here are described with too little detail to be useful to the reader who is not already familiar. I would suggest either taking them out and just referring to Runge (2018) or describing each one briefly.
– Table 1: What does "Use of iterative conditioning" mean? This should be clarified in the text.
– L262–263: How are lags incorporated into LKIF? This needs to be described in greater detail since it is used in the experiments.
– Figure 1e: the analytical values for x1->x1 and x2->x2 are excluded here. Why is this?
– L295–296: In almost all cases, LKIF seems to suggest a strong self-influence even when this is not present. Do you have any explanation of this effect? Perhaps it would be good to recommend against using LKIF for detecting self-influences.
– Figure 2d: Can you please add the self-influences to this graph (as arrows from a node to itself), as well as all the other graphs?
– Figure 3: It would be easy to add the false negatives to these plots by having white squares with a blue (or black) rectangle for those links that exist but were not detected by a given method. Can you please do this for all the plots?
– Table 2: It would be nice to add something like the phi coefficient (https://en.wikipedia.org/wiki/Phi_coefficient) for each case and method, to summarize the performance with a single number.
– Discussion: This paper does not test the robustness of the methods to noise, which is a crucial consideration in practice. This could be stated as another area for future work.Minor:
– L36: "Taken's theorem" -> "Takens's theorem" or "Takens' theorem" (the namesake is Takens, not Taken)
– L222: "such has" -> "such as"References
– Liang, X. S., & Kleeman, R. (2005). Information Transfer between Dynamical System Components. Physical Review Letters, 95(24), 244101. https://doi.org/10.1103/PhysRevLett.95.244101
– McGraw, M. C., & Barnes, E. A. (2018). Memory Matters: A Case for Granger Causality in Climate Variability Studies. Journal of Climate, 31(8), 3289–3300. https://doi.org/10.1175/JCLI-D-17-0334.1Citation: https://doi.org/10.5194/egusphere-2023-2212-RC1 -
RC2: 'Comment on egusphere-2023-2212', Anonymous Referee #2, 18 Dec 2023
The manuscript presents an interesting comparison of the output of two causality-detection methods, LKIF and PCMCI, on synthetic data generated by models of different characteristics, and real data (a set of relevant climatic indices). The manuscript is clearly written and the results are sound. Therefore, I am happy to recommend the acceptance of this manuscript, with a minor optional suggestion.
In the abstract, the authors say "Detecting causal links from the fourth model is more challenging as the system is nonlinear and chaotic." Model 4 is the well-known 3D Lorenz (1963) model, while Model 2 is a linear 6D model, and Model 3 is a 9D nonlinear model. Can the authors comment on the role of the model's dimensionality? How the dimensionality of the model affects the performance? Which method can be expected to provide more accurate results, in the case of high dimensional systems?
Citation: https://doi.org/10.5194/egusphere-2023-2212-RC2 - AC1: 'Reply to both reviewers', David Docquier, 12 Jan 2024
Interactive discussion
Status: closed
-
RC1: 'Comment on egusphere-2023-2212', Anonymous Referee #1, 13 Nov 2023
The authors apply two causal inference methods to a variety of test models as well as a climate example. The paper is interesting and worthy of publication pending some revisions for clarity.
Comments:
– L25–7: Although instantaneous correlations cannot determine the causal direction, lagged regressions are often used for this purpose, despite their drawbacks compared to causal inference methods. It may be worth citing McGraw and Barnes (2018) here, since they support your point about the need for causality analysis rather than lagged regressions for climate studies.
– L92: The notation dw_k ∼ sqrt(dt) N(0, 1) is unusual. Can you please use the typical notation in which this is stated using non-infinitesimal increments, i.e., something like w_k(t+t_0) - w_k(t_0) ∼ sqrt(t) N(0, 1)?
– Eq. 5: In Liang and Kleeman (2005), they derive a different expression for discrete and continuous time. Do you need to use different expressions for LKIF here in either case?
– L211–2: There is not enough information about how the significance was computed. What is the null hypothesis, and how exactly was the bootstrap distribution used? Please add details. Also, the significance test for PCMCI is not described.
– L222–4: The conditions here are described with too little detail to be useful to the reader who is not already familiar. I would suggest either taking them out and just referring to Runge (2018) or describing each one briefly.
– Table 1: What does "Use of iterative conditioning" mean? This should be clarified in the text.
– L262–263: How are lags incorporated into LKIF? This needs to be described in greater detail since it is used in the experiments.
– Figure 1e: the analytical values for x1->x1 and x2->x2 are excluded here. Why is this?
– L295–296: In almost all cases, LKIF seems to suggest a strong self-influence even when this is not present. Do you have any explanation of this effect? Perhaps it would be good to recommend against using LKIF for detecting self-influences.
– Figure 2d: Can you please add the self-influences to this graph (as arrows from a node to itself), as well as all the other graphs?
– Figure 3: It would be easy to add the false negatives to these plots by having white squares with a blue (or black) rectangle for those links that exist but were not detected by a given method. Can you please do this for all the plots?
– Table 2: It would be nice to add something like the phi coefficient (https://en.wikipedia.org/wiki/Phi_coefficient) for each case and method, to summarize the performance with a single number.
– Discussion: This paper does not test the robustness of the methods to noise, which is a crucial consideration in practice. This could be stated as another area for future work.Minor:
– L36: "Taken's theorem" -> "Takens's theorem" or "Takens' theorem" (the namesake is Takens, not Taken)
– L222: "such has" -> "such as"References
– Liang, X. S., & Kleeman, R. (2005). Information Transfer between Dynamical System Components. Physical Review Letters, 95(24), 244101. https://doi.org/10.1103/PhysRevLett.95.244101
– McGraw, M. C., & Barnes, E. A. (2018). Memory Matters: A Case for Granger Causality in Climate Variability Studies. Journal of Climate, 31(8), 3289–3300. https://doi.org/10.1175/JCLI-D-17-0334.1Citation: https://doi.org/10.5194/egusphere-2023-2212-RC1 -
RC2: 'Comment on egusphere-2023-2212', Anonymous Referee #2, 18 Dec 2023
The manuscript presents an interesting comparison of the output of two causality-detection methods, LKIF and PCMCI, on synthetic data generated by models of different characteristics, and real data (a set of relevant climatic indices). The manuscript is clearly written and the results are sound. Therefore, I am happy to recommend the acceptance of this manuscript, with a minor optional suggestion.
In the abstract, the authors say "Detecting causal links from the fourth model is more challenging as the system is nonlinear and chaotic." Model 4 is the well-known 3D Lorenz (1963) model, while Model 2 is a linear 6D model, and Model 3 is a 9D nonlinear model. Can the authors comment on the role of the model's dimensionality? How the dimensionality of the model affects the performance? Which method can be expected to provide more accurate results, in the case of high dimensional systems?
Citation: https://doi.org/10.5194/egusphere-2023-2212-RC2 - AC1: 'Reply to both reviewers', David Docquier, 12 Jan 2024
Peer review completion
Journal article(s) based on this preprint
Model code and software
Codes to compute Liang index and correlation for comparison study David Docquier https://doi.org/10.5281/zenodo.8383534
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 259 | 120 | 23 | 402 | 8 | 11 |
- HTML: 259
- PDF: 120
- XML: 23
- Total: 402
- BibTeX: 8
- EndNote: 11
Viewed (geographical distribution)
| Country | # | Views | % |
|---|---|---|---|
| United States of America | 1 | 104 | 26 |
| China | 2 | 59 | 15 |
| Germany | 3 | 45 | 11 |
| Belgium | 4 | 36 | 9 |
| Netherlands | 5 | 15 | 3 |
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
- 104
Cited
1 citations as recorded by crossref.
Giorgia Di Capua
Reik V. Donner
Carlos A. L. Pires
Amélie Simon
Stéphane Vannitsem
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(2072 KB) - Metadata XML