Preprints
https://doi.org/10.5194/egusphere-2022-152
https://doi.org/10.5194/egusphere-2022-152
 
27 Apr 2022
27 Apr 2022

The ICON-A model for direct QBO simulations on GPUs (version icon-cscs:baf28a514)

Marco A. Giorgetta1, William Sawyer2, Xavier Lapillonne3, Panagiotis Adamidis4, Dmitry Alexeev5, Valentin Clément6, Remo Dietlicher3, Jan Frederik Engels4, Monika Esch1, Henning Franke1,12, Claudia Frauen4, Walter M. Hannah7, Benjamin R. Hillman8, Luis Kornblueh1, Philippe Marti6, Matthew R. Norman9, Robert Pincus10, Sebastian Rast1, Daniel Reinert11, Reiner Schnur1, Uwe Schulzweida1, and Bjorn Stevens1 Marco A. Giorgetta et al.
  • 1Max-Planck-Institut für Meteorologie, Hamburg, Germany
  • 2Centro Svizzero di Calcolo Scientifico, Lugano, Switzerland
  • 3Bundesamt für Meteorologie und Klimatologie MeteoSchweiz, Zürich-Flughafen, Switzerland
  • 4Deutsches Klimarechenzentrum, Hamburg, Germany
  • 5NVIDIA, Zürich, Switzerland
  • 6Center for Climate Systems Modeling, ETH, Zürich, Switzerland
  • 7Lawrence Livermore National Laboratory, Livermore, USA
  • 8Sandia National Laboratories, Albuquerque, USA
  • 9Oak Ridge National Laboratory, Oak Ridge, USA
  • 10Lamont-Doherty Earth Observatory, Columbia University, Palisades, USA
  • 11Deutscher Wetterdienst, Offenbach, Germany
  • 12International Max Planck Research School on Earth System Modelling, Hamburg, Germany

Abstract. Classical numerical models for the global atmosphere, as used for numerical weather forecasting or climate research, have been developed for conventional central processing unit (CPU) architectures. This now hinders the employment of such models on current top performing supercomputers, which achieve their computing power with hybrid architectures, mostly using graphics processing units (GPUs). Thus also scientific applications of such models are restricted to the lesser computer power of CPUs. Here we present the development of a GPU enabled version of the ICON atmosphere model (ICON-A) motivated by a research project on the quasi-biennial oscillation (QBO), a global scale wind oscillation in the equatorial stratosphere that depends on a broad spectrum of atmospheric waves, which origins from tropical deep convection. Resolving the relevant scales, from a few km to the size of the globe, is a formidable computational problem, which can only be realized now on top performing supercomputers. This motivated porting ICON-A, in the specific configuration needed for the research project, in a first step to the GPU architecture of the Piz Daint computer at the Swiss National Supercomputing Centre, and in a second step to the Juwels-Booster computer at the Forschungszentrum Jülich. On Piz Daint the ported code achieves a single node GPU vs. CPU speed-up factor of 6.3, and now allows global experiments at a horizontal resolution of 5 km on 1024 computing nodes with 1 GPU per node with a turnover of 48 simulated days per day. On Juwels-Booster the more modern hardware in combination with an upgraded code base allows for simulations at the same resolution on 128 computing nodes with 4 GPUs per node and a turnover of 133 simulated days per day. Additionally, the code still remains functional on CPUs as it is demonstrated by additional experiments on the Levante compute system at the German Climate Computing Center. While the application shows good weak scaling making also higher resolved global simulations possible, the strong scaling on GPUs is relatively weak, which limits the options to increase turnover with more nodes. Initial experiments demonstrate that the ICON-A model can simulate downward propagating QBO jets, which are driven by wave meanflow interaction.

Journal article(s) based on this preprint

16 Sep 2022
The ICON-A model for direct QBO simulations on GPUs (version icon-cscs:baf28a514)
Marco A. Giorgetta, William Sawyer, Xavier Lapillonne, Panagiotis Adamidis, Dmitry Alexeev, Valentin Clément, Remo Dietlicher, Jan Frederik Engels, Monika Esch, Henning Franke, Claudia Frauen, Walter M. Hannah, Benjamin R. Hillman, Luis Kornblueh, Philippe Marti, Matthew R. Norman, Robert Pincus, Sebastian Rast, Daniel Reinert, Reiner Schnur, Uwe Schulzweida, and Bjorn Stevens
Geosci. Model Dev., 15, 6985–7016, https://doi.org/10.5194/gmd-15-6985-2022,https://doi.org/10.5194/gmd-15-6985-2022, 2022
Short summary

Marco A. Giorgetta et al.

Interactive discussion

Status: closed

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on egusphere-2022-152', Anonymous Referee #1, 27 May 2022
    • AC1: 'Reply on RC1', Marco Giorgetta, 07 Jul 2022
  • RC2: 'Comment on egusphere-2022-152', Italo Epicoco, 13 Jul 2022
    • AC2: 'Reply on RC2', Marco Giorgetta, 20 Jul 2022

Interactive discussion

Status: closed

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • RC1: 'Comment on egusphere-2022-152', Anonymous Referee #1, 27 May 2022
    • AC1: 'Reply on RC1', Marco Giorgetta, 07 Jul 2022
  • RC2: 'Comment on egusphere-2022-152', Italo Epicoco, 13 Jul 2022
    • AC2: 'Reply on RC2', Marco Giorgetta, 20 Jul 2022

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision
AR by Marco Giorgetta on behalf of the Authors (09 Aug 2022)  Author's response    Author's tracked changes    Manuscript
ED: Publish subject to technical corrections (26 Aug 2022) by Sophie Valcke
AR by Marco Giorgetta on behalf of the Authors (26 Aug 2022)  Author's response    Manuscript

Journal article(s) based on this preprint

16 Sep 2022
The ICON-A model for direct QBO simulations on GPUs (version icon-cscs:baf28a514)
Marco A. Giorgetta, William Sawyer, Xavier Lapillonne, Panagiotis Adamidis, Dmitry Alexeev, Valentin Clément, Remo Dietlicher, Jan Frederik Engels, Monika Esch, Henning Franke, Claudia Frauen, Walter M. Hannah, Benjamin R. Hillman, Luis Kornblueh, Philippe Marti, Matthew R. Norman, Robert Pincus, Sebastian Rast, Daniel Reinert, Reiner Schnur, Uwe Schulzweida, and Bjorn Stevens
Geosci. Model Dev., 15, 6985–7016, https://doi.org/10.5194/gmd-15-6985-2022,https://doi.org/10.5194/gmd-15-6985-2022, 2022
Short summary

Marco A. Giorgetta et al.

Model code and software

The ICON-A model for direct QBO simulations on GPUs Marco A. Giorgetta https://doi.org/10.17617/3.5CYUFN

Marco A. Giorgetta et al.

Viewed

Total article views: 876 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
640 225 11 876 8 6
  • HTML: 640
  • PDF: 225
  • XML: 11
  • Total: 876
  • BibTeX: 8
  • EndNote: 6
Views and downloads (calculated since 27 Apr 2022)
Cumulative views and downloads (calculated since 27 Apr 2022)

Viewed (geographical distribution)

Total article views: 697 (including HTML, PDF, and XML) Thereof 697 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 16 Sep 2022
Download

The requested preprint has a corresponding peer-reviewed final revised paper. You are encouraged to refer to the final revised version.

Short summary
This work presents a first version of the ICON atmosphere model that works not only on CPUs but also on GPUs. This GPU enabled ICON version is benchmarked on two GPU machines and a new CPU machine. While the weak scaling is very good on CPUs and GPUs, the strong scaling is poor on GPUs. But the high performance of GPU machines allowed first simulations of a short period of the quasi-biennial oscillation at very high resolution with explicit convection and gravity wave forcing.