the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 20 Mar 2025
A Python diagnostics package for evaluation of MJO-Teleconnections in S2S forecast systems
Abstract. The MJO-Teleconnections diagnostics package is an open-source Python software package that provides process-level evaluation of MJO-Teleconnections predicted by subseasonal-to-seasonal (S2S) forecast systems. The package provides in-depth process-level evaluation of both tropospheric and stratospheric pathways defining the atmospheric teleconnections from the tropics to extratropics on S2S times scale. The analyses include comparison of a forecast model with a default verification data set or user-provided verification data. The package consists of a user-friendly graphic user interface (GUI), which allows the package to be applied to both operational and research models. This approach allows for efficient data management and reproducibility of analysis.
- Preprint
(18277 KB) - Metadata XML
- BibTeX
- EndNote
Status: open (until 15 May 2025)
-
RC1: 'Comment on egusphere-2025-1142', Anonymous Referee #1, 17 Apr 2025
reply
Review of “A Python diagnostics package for evaluation of MJO-Teleconnections in S2S forecast systems” by Stan et al.
Recommendation: Major revisions
Summary:
The manuscript describes development of a Python-based process-oriented diagnostics package to assess the quality of MJO teleconnections in subseasonal forecast systems. Both tropospheric and stratospheric pathways are examined, which rely on diagnostics already developed in the published literature. Description of the GUI interface for this package is also presented.
The diagnostic and interface described have the potential to be an extremely useful package to diagnose MJO teleconnection performance in forecasting models that will be adopted by forecasting centers. Importantly, the diagnostics presented in this package are justified based on documentation in the peer-reviewed literature and physically sound. I commend the authors for the amount of work required to generate a package such as this, as it is important work and no small task. That said, the paper requires major revisions before acceptance, and many opportunities are missed to provide a much more compelling description and advertisement of this effort that will enhance uptake.
First, there are many strong individual contributions plugged into this paper, but the paper doesn’t flow well, doesn’t have a unified voice, and is disjointed. One or two authors need to take the initiative to provide such a cohesive narrative. In places, the manuscript reads like a collection of parts, and more effort is needed to make the whole greater than the sum of parts. Second, results from a model are presented alongside observations analysis, but little if any insight is ever provided into the nature of deficiencies in the model and why the particular diagnostic being examined provides insight into model performance. The model results just seem like extra figures that are not discussed. Third, the summary section is disappointingly terse and fragmented, and misses an opportunity for a more thorough and enlightening discussion of limitations, challenges, and where the package will develop in the future. This section reads like an afterthought that was written very quickly. Lastly, more details on format of data inputs might be provided. Further comments are listed below.
Comments:
- Lines 52-55. This description of the mechanism through which the MJO produces a stationary Rossby wave teleconnection is terse and might be expanded, given the emphasis of the paper.
- In general, I find the writing in the introduction to be choppy and not flow well, and so could use some work to improve the narrative.
- Many challenges arise when developing a diagnostic package for community use, for example:
Neelin, J. D., J. P. Krasting, A. Radhakrishnan, J. Liptak, T. Jackson, Y. Ming, W. Dong, A. Gettelman, D. R. Coleman, E. D. Maloney, A. A. Wing, Y.-H. Kuo, F. Ahmed, P. Ullrich, C. M. Bitz, R. B. Neale, A. Ordonez, and Elizabeth A. Maroon, 2023: Process-oriented diagnostics: principles, practice, community development and common standards. Bull. Amer. Meteor. Soc., 104, E1452–E1468.
It would be worthwhile to acknowledge this and also discuss how this diagnostic package overcomes or hopes to overcome such challenges. A lot can go wrong with implementation of a package like this that seems simple on its face, but is complicated to implement in detail. As such, it would be good for this group to interface with the NOAA Model Diagnostics Task Force to share experiences and lessons.
- Lines 120-131. The devil may be in the details and a lot can go wrong in this standardization of data. Some discussion of lessons learned when different models are entrained, as well as discussion of the importance of standardization (e.g. Neelin et al. 2023) might be helpful. There is a lot unclear about the details, including what temporal resolution forecast data can be used, what number of forecast lags can be used, etc.
- Figures 3 and 4. No interpretation of what these figures are telling us about model performance is provided. For example, is the bottom row of Figure 3 concerning, good, or somewhere in between regarding model performance? Providing evidence that these diagnostics are insightful will help uptake of the package.
- Figure 6. Again, a bit more insight into model deficiencies that this diagnostic provides would be helpful.
- Figure 8. Again here, what insight is provided into this particular model by examining the right column of this figure? It seems unnecessary to include panels from a model in this paper if they are not discussed and interpreted.
- Section: 2.2.6 MJO. This section provides information on basic MJO performance, and seems out of order relative to the rest of the plots that discuss teleconnections. This section might be moved to be the first diagnostic discussed. This relates to my comment above about the manuscript not flowing well.
- Summary, limitations and future development. I was expecting a more thorough and earnest discussion of the limitations of the manuscript and future developments of the package, but this section is disappointingly terse and cursory, with fragments of ideas. The ideas in this section need to be expanded.
Citation: https://doi.org/10.5194/egusphere-2025-1142-RC1
Model code and software
GitHub Repository Cristiana Stan, Saisri Kollapaneni, Andrea M. Jenney, Jiabao Wang, Zheng Wu, Cheng Zheng Hyemi Kim, Chaim I. Garfinkel, and Ayush Singh https://doi.org/10.5281/zenodo.15002615
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 97 | 24 | 4 | 125 | 5 | 5 |
- HTML: 97
- PDF: 24
- XML: 4
- Total: 125
- BibTeX: 5
- EndNote: 5
Viewed (geographical distribution)
| Country | # | Views | % |
|---|---|---|---|
| United States of America | 1 | 67 | 41 |
| China | 2 | 20 | 12 |
| Germany | 3 | 15 | 9 |
| undefined | 4 | 13 | 8 |
| France | 5 | 8 | 4 |
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
- 67