the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Physics-motivated Cell-octree Adaptive Mesh Refinement in the Vlasiator 5.3 Global Hybrid-Vlasov Code
Abstract. Automatically adaptive grid resolution is a common way of improving simulation accuracy while keeping the computational efficiency at a manageable level. In space physics adaptive grid strategies are especially useful as simulation volumes are extreme, while the most accurate physical description is based on electron dynamics and hence requires very small grid cells and time steps. Therefore, many past global simulations encompassing e.g. the near-Earth space have made tradeoffs in terms of the physical description and used laws of magnetohydrodynamics (MHD) that require less accurate grid resolutions. Recently, using supercomputers, it has become possible to model the near-Earth space domain with an ion-hybrid scheme going beyond the MHD-based fluid dynamics. These simulations, however, must develop a new adaptive mesh strategy beyond what is used in MHD simulations.
We developed an automatically adaptive grid refinement strategy for ion-hybrid Vlasov schemes, and implemented it within the Vlasiator global solar wind – magnetosphere – ionosphere simulation Vlasiator. This method automatically adapts the resolution of the Vlasiator grid using two indices: one formed as a maximum of dimensionless gradients measuring the rate of spatial change in selected variables, and the other derived from the ratio of the current density to the magnetic field density perpendicular to the current. Both these indices can be tuned independently to reach a desired level of refinement and computational load. We test the indices independently and compare the results to a control run using static refinement.
The results show that adaptive refinement highlights relevant regions of the simulation domain and keeps the computational effort at a manageable level. We find that the refinement shows some overhead in rate of cells solved per second. This overhead can be large compared to the control run without adaptive refinement, possibly due to resource utilisation, grid complexity and issues in load balancing. These issues lay a development roadmap for future optimisations.
- Preprint
(1816 KB) - Metadata XML
- BibTeX
- EndNote
Status: open (until 03 May 2024)
-
RC1: 'Comment on egusphere-2024-301', Anonymous Referee #1, 16 Apr 2024
reply
Line 64: Does Vlasiator use the real mass ratio?
Line 70: Please elaborate more about what spatial optimizations are needed.
Line 84: AMR is not used in MHD-AEPIC. MHD-AEPIC is dynamically apply kinetic physics in regions that are needed. The grid resolution is not varying in space. BATS-R-US indeed uses AMR in the MHD grid. I suggest removing the MHD-AEPIC citation here.
Figure 5(a): Why the cell number increases for the unrefined run?
Table 1: Please list the total runtime of each run in the table, that would justify using AMR in the future production runs, even the scaling seems to be bad.
Citation: https://doi.org/10.5194/egusphere-2024-301-RC1 -
RC2: 'Comment on egusphere-2024-301', Anonymous Referee #2, 16 Apr 2024
reply
I have read the manuscript "Physics-motivated Cell-octree Adaptive Mesh
Refinement in the Vlasiator 5.3 Global Hybrid-Vlasov Code" authored by
Kotipalo et al. The manuscript introduces a new adaptive mesh refinement
strategy in the kinetic hybrid-Vlasov code, Vlasiator. A series of test runs
were carried to compare the new method with the existing static mesh
refinement approach. Although, the performance of the new method is not
perfect, the overall results are interesting, and the new strategy warrants
further investigation. Beyond some minor comments, I would recommend that
this paper be accepted for publication in GMD.Was OpenMP used in the test runs? Regardless of the answer, could the
authors provide additional discussion on its impact on performance,
particularly in terms of load balancing?It is speculated that the load balance was non-ideal because the weights
used in the load balancing algorithm did not account for changes in the
mesh. Would more frequent load balancing help mitigate this issue?I noticed in the Acknowledgements that the simulations were performed on CSC
Mahti supercomputer. However, while reading the manuscript, I had questions
about this. It would help the reader if a brief description of the computer
were included alongside the presentation of the test results.Line 138, starting with 'These gradients are': Technically speaking, an
expression like (delta B)^2 / (2 mu_0) is not the gradient of the magnetic
field energy, which would be B delta B / mu_0. To avoid confusion, could the
authors revise this sentence?There is a typo in Line 173. `his` -> `this`?
Citation: https://doi.org/10.5194/egusphere-2024-301-RC2 -
RC3: 'Comment on egusphere-2024-301', Anonymous Referee #3, 17 Apr 2024
reply
General comments
This paper describes the implementation of adaptive mesh refinement in the Vlasiator hybrid code. The authors use a series of metrics previously used in MHD codes to prescribe where refinement takes place, and perform a series of simulations, providing statistics on the computational efficiency. While the implementation is a useful contribution, and the computational sections are well-written, the paper requires revision before it can be published due to missing physics information.
Specific comments
1) Although this is primarily a numerical paper, there is a lack of physics information on the system being studied. This inculdes
- Line 52 - what is the ion kinetic scale compared to the resolution?
- What are the physical parameters of the simulation being used for the study (eg plasma and field parameters)2) Figure 1 - What are the units in the colour plot? Also, the physics appears different in the two runs (the most obvious difference is the flux rope in the tail)?
3) Line 239 - Should this be XY and XZ planes?
4) Line 245 - What do the authors mean when referring to "foreshocks"? These do not appear to be the perturbations upstream of the quasi-parallel shock, which is the usual definition.
5) Figure 5 - The bar charts in (c)-(j) have some quantities obscured due to the overlapping. I would suggest modifying the opacity or a line plot.
6) Figure 6 - It would be helpful to provide a plot of the control in addition to the existing plots.
7) In general, the authors do not show that their simulations converge. This is alluded to in point 2), where the behaviour of the magnetotail looks different in the two runs. Without this information, whether the method works cannot be judged.
8) Table 3 - I suggest providing the total run time in the table as well, since that is of practical use.
Suggestions
1) Line 179 - Please define induced refinement.
2) I would suggest using a different font/italics for dccrg
Technical corrections1) Line 138 - move (a-e) to before the quantities. E.g (a) particle density ....
2) Line 173 - his -> this
3) There are certain parts of the text where contractions are used (e.g don't). These do not fit stylistically with the rest of the paper.Citation: https://doi.org/10.5194/egusphere-2024-301-RC3
Data sets
AMR Test Configuration Leo Kotipalo https://doi.org/10.23729/f7f9d95d-1e23-49e3-9c35-1d8e26c47bf7
Model code and software
fmihpc/vlasiator: Vlasiator 5.3 Yann Pfau-Kempf, Sebastian von Alfthan, Urs Ganse, Markus Battarbee, Leo Kotipalo, Tuomas Koskela, Ilja, Arto Sandroos, Kostis Papadakis, Markku Alho, Hongyang Zhou, Miro Palmu, Maxime Grandin, Jonas Suni, Dimitry Pokhotelov, and Konstatinos Horaites https://doi.org/10.5281/zenodo.10600112
Viewed
HTML | XML | Total | BibTeX | EndNote | |
---|---|---|---|---|---|
169 | 29 | 15 | 213 | 6 | 6 |
- HTML: 169
- PDF: 29
- XML: 15
- Total: 213
- BibTeX: 6
- EndNote: 6
Viewed (geographical distribution)
Country | # | Views | % |
---|
Total: | 0 |
HTML: | 0 |
PDF: | 0 |
XML: | 0 |
- 1