The 4-mode Modal Aerosol Module in C++ (MAM4xx) v1.0: Representing Prognostic Aerosols in a Global Cloud-System Resolving Atmosphere Model for GPU Exascale Computing

Abstract. Aerosols are a key component of Earth system models since they affect meteorology and the Earth’s energy budget through complex cloud-aerosol-radiation interactions. Over the past decade, Earth system models have increased their spatial resolution to better resolve atmospheric processes; however, this advancement comes with significantly higher computational costs. To address this issue, some models now take advantage of high-performance Graphics Processing Unit (GPU) exascale computer clusters which offer faster processing capabilities with a higher level of parallelism than traditional Central Processing Unit (CPU) systems, but these models still lack detailed representations of aerosols. In this study, we describe the development of a new GPU-enabled prognostic aerosol model based on the four-mode version of the Modal Aerosol Module (MAM4), called MAM4xx, that has been coupled to the Energy Exascale Earth System (E3SM) Atmospheric Model (EAM) in C++ (EAMxx). To the best of our knowledge, MAM4xx is the first fully GPU-enabled aerosol model with sophisticated process representations. MAM4 has been completely rewritten in C++ using the Kokkos performance-portability programming library while preserving all the physical and chemical processes in the original Fortran version. The Kokkos library ensures compatibility across GPUs from various vendors and thus enables execution on multiple GPU exascale high-performance computer clusters. We describe the steps undertaken to port the code to C++/Kokkos as well as the rigorous testing methodology (i.e., unit tests, real-world tests) so that the functionality remains intact and that bugs were not inadvertently introduced. We demonstrate that MAM4xx coupled within EAMxx with ~12-km horizontal grid spacing behaves as expected for real-world conditions based on comparison with observations and a reanalysis aerosol dataset over the central U.S. during the spring of 2016. Currently, MAM4xx increases the computational cost of the host atmospheric model by ~40%, which is due primarily to the treatment of aerosol-cloud interactions, highlighting the need to optimize these processes for GPU computational efficiency. Future improvements to MAM4xx will benefit from recent advancements in the physical representation of aerosols in MAM4 as they are ported to C++/Kokkos. Additional testing for longer periods of time to encompass a wider range of atmospheric conditions, over other geographic regions, and at higher spatial resolution will be conducted in the future to more robustly assess simulated aerosol properties.