the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 13 Jun 2023
Analysis of geomagnetic observatory data and detection of geomagnetic jerks with the MOSFiT software package
Abstract. MOSFiT (Magnetic Observatories and Stations Filtering Tool) is a python package to visualize and filter data from magnetic observatories and magnetometer stations. The purpose of MOSFiT is to automatically isolate and analyze the secular variation (SV) information measured by geomagnetic observatories data. External field contributions may be reduced by selecting data according to local time and geomagnetic indices and by subtracting the magnetospheric field predictions of the CHAOS-7 model. MOSFiT calculates the SV by annual differences of monthly means and geomagnetic jerk occurrence time and amplitude are automatically calculated by fitting two straight-line segments in a user-defined time interval of the SV time series. Here, we present the new python package, validate it against independent results from previous publications, and show its application. In particular, we quantify the RMS misfit between SV derived from processing schemes and the SV predicted by CHAOS-7. Analysing the INTERMAGNET quasi-definitive data with MOSFIT allows for a timely investigation of SV such as the detection of recent geomagnetic jerks. It can also be used for data selection for e.g., external field studies as well as for quality control of geomagnetic observatory data.
-
Notice on discussion status
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
-
Preprint
(4807 KB)
-
Supplement
(4066 KB)
-
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(4807 KB) - Metadata XML
-
Supplement
(4066 KB) - BibTeX
- EndNote
- Final revised paper
Journal article(s) based on this preprint
Interactive discussion
Status: closed
-
CC1: 'Comment on egusphere-2023-79', J. Miquel Torta, 14 Jun 2023
To validate the methods provided in this manuscript, in Sect. 4.2 we compare the automatic detection of the 2007, 2011 and 2014 jerks using observatory data from NGK, EBR, TAM and ASC with that given by Torta et al. (2015). The comparison is presented in Table 4. The results show a very good agreement for the time of occurrence of the jerks as well as for the amplitudes at all four observatories except for the 2011 time of occurrence jerk reported by Torta et al. for ASC, which was 2012.0. However Fig. 6 of this manuscript, and indeed Fig 1 of Torta et al. (2015), indicate that both the observatory data and the CHAOS model prediction show a jerkk around 2010. After reviewing our results and notes we can confirm that in fact the occurrence found with our method was 2010.2 (which is very close to the result given in this manuscript), but due to a transcription error we wrote 2012.0. We would be grateful if the authors could indicate this (e.g. as a personal communication) in any revisions to their original manuscript.
J. M. Torta
Citation: https://doi.org/10.5194/egusphere-2023-79-CC1 -
RC1: 'Comment on egusphere-2023-79', Seiki Asari, 16 Aug 2023
Basically I find the paper worth publishing, in which a new python tool MOSFiT for magnetic observatory data analysis is introduced. As the source program has been made open, it can be used by observers and researchers widely for different purposes, not just for secular variation analysis but for data processing as well. To improve the manuscript before publishing, I would give a suggestion as below, for which some further computation (but not tough) is needed to supplement their quantitative results. I would like the authors to consider it and make a necessary revision.
On top of its useful function for jerk detection, MOSFiT is featured with various methods for external field mitigation. To characterize the different methods more in detail, I suggest that quantitative comparison of SV misfits derived with all those methods be made in Section 4.1, where only two cases (CHAOS-7 corrected and uncorrected SVs) are compared in the manuscript. Analyses for KP, QD and NT method, or even for a combination of their use with the CHAOS-7 method, can also be made to complement Table 3 with their outcomes. Furthermore, Figure 9 and the text for its explanation in Section 5 may be moved to Section 4.1, as a qualitative illustration to support the quantitative analyses, which can be referenced from Section 5 now dedicated solely for jerk discussion. (Note there is redundancy in L229-231 for describing Fig.9. Some have already been stated previously in L225- 229)
It is often observed that K-indices are rather small in recovery phase of storms, when the ring currents remain, still lowering X-component level. Therefore I imagine, a combination of the KP-method and CHOAS-7 method (or otherwise a selection with Dst-index) would be even more effective for excluding external fields from data.
Typos:
L15 variesranges
L17 Abrupt changes in SV ‘trend’ (as described properly in L145)
L118 can be can be
L193 latest the end
S. Asari
Citation: https://doi.org/10.5194/egusphere-2023-79-RC1 - AC1: 'Reply on RC1', Marcos Vinicius Siqueira da Silva, 20 Sep 2023
-
RC2: 'Comment on egusphere-2023-79', Jan Reda, 17 Aug 2023
GENERAL COMMENTS
The authors present their experiences related to the analysis of jerks, which are unpredictable events that suddenly speed up secular variations (SV) of the Earth's magnetic field.
Identifying jerks is not an easy task. The authors of this article undertook the difficult task of creating generally available software that performs such a task. The software, called MOSFIT (Magnetic Observatories and Stations Filtering Tool), works with geomagnetic data of Definitive or Quasi-Definitive status in IAGA-2002 format, which are easily available, for example, on the WDC (World Data Center) Edinburgh server. This applies to both INTERMAGNET observatory data and other observatories.
The MOSFIT package can also be used for quality control of data from geomagnetic observatories. The software has filtering, selection, and data visualization capabilities. MOSFIT was written in Python language, which is becoming increasingly popular. The authors provide a link to the software, as well as a guide to facilitate the use of the software package.
In the first chapters, the authors introduce the reader well to the subject of jerks and the problems that arise when detecting them. This is about isolating SV from final recordings of the entire geomagnetic field that also contain the influence of the external field. This is really a big challenge, because not all jerks are very expressive.
In order to isolate jerks, the package offers Hampel filtering as the first step. In the next stage, the contribution of the external field can be reduced by selecting data according to local midnight, Kp geomagnetic indices, quiet days, disturbed days, and by subtracting the magnetospheric field based on the CHAOS-7 model.
It is worth emphasizing that the package offers the possibility of visualization at every stage of work.
The authors presented the detection of jerks from 2007, 2011, and 2014 and compared the results with an analysis performed and published by another researcher. Generally, it can be stated that the consistency of detection was very good, both in terms of the time of occurrence of the jerk and its amplitude.
In summary, I am happy to recommend the manuscript for publication after making minor corrections listed below, in sections SPECIFIC COMMENTS and TECHNICAL, LANGUAGE AND OTHER REMARKS.
SPECIFIC COMMENTS
In the bibliography (References), it would be good to add DOI (Digital Object Identifier) identifiers if they are available. DOI identifiers are important because they add credibility to the source and make it easier to access publications. Currently, providing DOI’s is basically a standard in bibliographies.
TECHNICAL, LANGUAGE AND OTHER REMARKS
Lines 1,7,44,155,236 The word python should be rather capitalized in this context.
Line 15 There is missing space between “varies” and “ranges”
Line 39 Rather should be “close-to-final” (missing ”-“)
Line 58 “computacional” -> “computational”
Line 126 There is missing space between “time” and “series”
Lines 174,183 “table” -> “Table”
Line 232 “Fig 8” -> “Fig. 8” (missing dot)
Line 236 Should be rather “one-minute”
Line 282 Change the capitalisation: Journal of the American Statistical
Association
Line 310 Change the captalisation: Generalized Hampel Filters
Citation: https://doi.org/10.5194/egusphere-2023-79-RC2 - AC2: 'Reply on RC2', Marcos Vinicius Siqueira da Silva, 20 Sep 2023
-
RC3: 'Comment on egusphere-2023-79', Rudi Cop, 18 Aug 2023
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-79/egusphere-2023-79-RC3-supplement.pdf
- AC3: 'Reply on RC3', Marcos Vinicius Siqueira da Silva, 20 Sep 2023
Interactive discussion
Status: closed
-
CC1: 'Comment on egusphere-2023-79', J. Miquel Torta, 14 Jun 2023
To validate the methods provided in this manuscript, in Sect. 4.2 we compare the automatic detection of the 2007, 2011 and 2014 jerks using observatory data from NGK, EBR, TAM and ASC with that given by Torta et al. (2015). The comparison is presented in Table 4. The results show a very good agreement for the time of occurrence of the jerks as well as for the amplitudes at all four observatories except for the 2011 time of occurrence jerk reported by Torta et al. for ASC, which was 2012.0. However Fig. 6 of this manuscript, and indeed Fig 1 of Torta et al. (2015), indicate that both the observatory data and the CHAOS model prediction show a jerkk around 2010. After reviewing our results and notes we can confirm that in fact the occurrence found with our method was 2010.2 (which is very close to the result given in this manuscript), but due to a transcription error we wrote 2012.0. We would be grateful if the authors could indicate this (e.g. as a personal communication) in any revisions to their original manuscript.
J. M. Torta
Citation: https://doi.org/10.5194/egusphere-2023-79-CC1 -
RC1: 'Comment on egusphere-2023-79', Seiki Asari, 16 Aug 2023
Basically I find the paper worth publishing, in which a new python tool MOSFiT for magnetic observatory data analysis is introduced. As the source program has been made open, it can be used by observers and researchers widely for different purposes, not just for secular variation analysis but for data processing as well. To improve the manuscript before publishing, I would give a suggestion as below, for which some further computation (but not tough) is needed to supplement their quantitative results. I would like the authors to consider it and make a necessary revision.
On top of its useful function for jerk detection, MOSFiT is featured with various methods for external field mitigation. To characterize the different methods more in detail, I suggest that quantitative comparison of SV misfits derived with all those methods be made in Section 4.1, where only two cases (CHAOS-7 corrected and uncorrected SVs) are compared in the manuscript. Analyses for KP, QD and NT method, or even for a combination of their use with the CHAOS-7 method, can also be made to complement Table 3 with their outcomes. Furthermore, Figure 9 and the text for its explanation in Section 5 may be moved to Section 4.1, as a qualitative illustration to support the quantitative analyses, which can be referenced from Section 5 now dedicated solely for jerk discussion. (Note there is redundancy in L229-231 for describing Fig.9. Some have already been stated previously in L225- 229)
It is often observed that K-indices are rather small in recovery phase of storms, when the ring currents remain, still lowering X-component level. Therefore I imagine, a combination of the KP-method and CHOAS-7 method (or otherwise a selection with Dst-index) would be even more effective for excluding external fields from data.
Typos:
L15 variesranges
L17 Abrupt changes in SV ‘trend’ (as described properly in L145)
L118 can be can be
L193 latest the end
S. Asari
Citation: https://doi.org/10.5194/egusphere-2023-79-RC1 - AC1: 'Reply on RC1', Marcos Vinicius Siqueira da Silva, 20 Sep 2023
-
RC2: 'Comment on egusphere-2023-79', Jan Reda, 17 Aug 2023
GENERAL COMMENTS
The authors present their experiences related to the analysis of jerks, which are unpredictable events that suddenly speed up secular variations (SV) of the Earth's magnetic field.
Identifying jerks is not an easy task. The authors of this article undertook the difficult task of creating generally available software that performs such a task. The software, called MOSFIT (Magnetic Observatories and Stations Filtering Tool), works with geomagnetic data of Definitive or Quasi-Definitive status in IAGA-2002 format, which are easily available, for example, on the WDC (World Data Center) Edinburgh server. This applies to both INTERMAGNET observatory data and other observatories.
The MOSFIT package can also be used for quality control of data from geomagnetic observatories. The software has filtering, selection, and data visualization capabilities. MOSFIT was written in Python language, which is becoming increasingly popular. The authors provide a link to the software, as well as a guide to facilitate the use of the software package.
In the first chapters, the authors introduce the reader well to the subject of jerks and the problems that arise when detecting them. This is about isolating SV from final recordings of the entire geomagnetic field that also contain the influence of the external field. This is really a big challenge, because not all jerks are very expressive.
In order to isolate jerks, the package offers Hampel filtering as the first step. In the next stage, the contribution of the external field can be reduced by selecting data according to local midnight, Kp geomagnetic indices, quiet days, disturbed days, and by subtracting the magnetospheric field based on the CHAOS-7 model.
It is worth emphasizing that the package offers the possibility of visualization at every stage of work.
The authors presented the detection of jerks from 2007, 2011, and 2014 and compared the results with an analysis performed and published by another researcher. Generally, it can be stated that the consistency of detection was very good, both in terms of the time of occurrence of the jerk and its amplitude.
In summary, I am happy to recommend the manuscript for publication after making minor corrections listed below, in sections SPECIFIC COMMENTS and TECHNICAL, LANGUAGE AND OTHER REMARKS.
SPECIFIC COMMENTS
In the bibliography (References), it would be good to add DOI (Digital Object Identifier) identifiers if they are available. DOI identifiers are important because they add credibility to the source and make it easier to access publications. Currently, providing DOI’s is basically a standard in bibliographies.
TECHNICAL, LANGUAGE AND OTHER REMARKS
Lines 1,7,44,155,236 The word python should be rather capitalized in this context.
Line 15 There is missing space between “varies” and “ranges”
Line 39 Rather should be “close-to-final” (missing ”-“)
Line 58 “computacional” -> “computational”
Line 126 There is missing space between “time” and “series”
Lines 174,183 “table” -> “Table”
Line 232 “Fig 8” -> “Fig. 8” (missing dot)
Line 236 Should be rather “one-minute”
Line 282 Change the capitalisation: Journal of the American Statistical
Association
Line 310 Change the captalisation: Generalized Hampel Filters
Citation: https://doi.org/10.5194/egusphere-2023-79-RC2 - AC2: 'Reply on RC2', Marcos Vinicius Siqueira da Silva, 20 Sep 2023
-
RC3: 'Comment on egusphere-2023-79', Rudi Cop, 18 Aug 2023
The comment was uploaded in the form of a supplement: https://egusphere.copernicus.org/preprints/2023/egusphere-2023-79/egusphere-2023-79-RC3-supplement.pdf
- AC3: 'Reply on RC3', Marcos Vinicius Siqueira da Silva, 20 Sep 2023
Peer review completion
Journal article(s) based on this preprint
Model code and software
Magnetic Observatories and Stations Filtering Tool (MOSFiT) Marcos Vinicius da Silva https://github.com/marcosv9/MOSFiT-package
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 297 | 96 | 32 | 425 | 40 | 14 | 17 |
- HTML: 297
- PDF: 96
- XML: 32
- Total: 425
- Supplement: 40
- BibTeX: 14
- EndNote: 17
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Marcos Vinicius da Silva
Katia Pinheiro
Achim Ohlert
The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.
- Preprint
(4807 KB) - Metadata XML
-
Supplement
(4066 KB) - BibTeX
- EndNote
- Final revised paper