the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 05 Sep 2024
SERGHEI v2.0: introducing a performance-portable, high-performance three-dimensional variably-saturated subsurface flow solver (SERGHEI-RE)
Abstract. Advances in high-performance parallel computing have significantly enhanced the speed of large-scale hydrological simulations. However, the diversity and rapid evolution of available computational systems and hardware devices limit model flexibility and increase code maintenance efforts. This paper introduces SERGHEI-RE, a three-dimensional, variably saturated subsurface flow simulator within the SERGHEI model framework. SERGHEI-RE adopts the Kokkos-based portable parallelization framework of SERGHEI, which facilitates scalability and ensures performance portability on various computational devices. Moreover, SERGHEI-RE provides options to solve the Richards Equation with iterative or non-iterative numerical schemes, enhancing model flexibility under complex hydrogeological conditions. The solution accuracy of SERGHEI-RE is validated using a series of benchmark tests, ranging from simple infiltration problems to practical hydrological, geotechnical and agricultural applications. The scalability and performance portability of SERGHEI-RE are demonstrated on a desktop workstation, as well as on multi-CPU and multi-GPU clusters, indicating that SERGHEI-RE is an efficient, scalable and performance portable tool for large-scale subsurface flow simulations.
- Preprint
(1172 KB) - Metadata XML
- BibTeX
- EndNote
Status: closed
-
RC1: 'Comment on egusphere-2024-2588', Anonymous Referee #1, 01 Oct 2024
In the manuscript titled “SERGHEI v2.0: introducing a performance-portable, high-performance three-dimensional variably-saturated subsurface flow solver (SERGHEI-RE)”, the authors develop the Richards Equation-based variably-saturated subsurface flow module of SERGHEI. The characteristics of the framework are accuracy, robustness, scalability, and portability, which preserves the parallel performance and scalability across different HPC hardware. There are still certain changes and clarifications that the authors should address prior to publication. For these reasons, I believe that the manuscript can be accepted for publication by the GMD after minor revision. Below, I have some general comments for the authors.
General comments:
- Line #48 – 49, it would be better provide further explanation on how parallel performance depends on HPC hardware. For example, are the differences in the parallel computing scripts caused by the different HPC compilers, scheduler, or are the differences due to the storage methods (e.g. HPSS, GPFS file system) used by the HPC? This would help readers quickly understand the practical problems that this study aims to solve. If only the compilation issues related to CPU parallelization and GPU parallelization are addressed (line #59-60), please appropriately discuss the limitations of this study.
- Line #124-129, it would be necessary to provide specific examples of the use of PC and MP schemes in SERGHEI-RE. This would make it more intuitive and help better understand the advanced nature of this research plan.
- Equation 8 could be illustrated with a diagram like Fig.2.
- The comparison figures of PC and MP show that, except for the 4 days simulation in Fig. 4, they almost completely overlap. The introduction section regarding these two different types of introductions cannot fully explain the outcome. Therefore, it is best to provide further explanation.
- Figure 11, almost all evaporation points are zero after 50 days, this seems to be caused by a model code error. If not, please explain briefly.
Specific comments:
- Figure 1, in this diagram, what is the difference between the dashed lines, solid lines, and those without borders?
- Line #100, what do α and represent?
Citation: https://doi.org/10.5194/egusphere-2024-2588-RC1 -
AC1: 'Reply on RC1', Zhi Li, 23 Oct 2024
We thank the reviewer for providing constructive comments on our work. Please find our response in the attachments.
Zhi Li
-
RC2: 'Reply on AC1', Anonymous Referee #1, 31 Oct 2024
I am sorry for the misunderstanding, the "script" in the first comment means the workflows to build the CPU/GPU binaries.
The reviewers have diligently addressed all the comments and revisions I provided during the previous review.Citation: https://doi.org/10.5194/egusphere-2024-2588-RC2
-
RC2: 'Reply on AC1', Anonymous Referee #1, 31 Oct 2024
-
RC3: 'Comment on egusphere-2024-2588', Anonymous Referee #2, 01 Dec 2024
The paper presents the development of a high-performance three-dimensional variably-saturated subsurface flow slover (SERGEI-RE). The contents is clear and shows good results. Likewise, the level of high performance computing is quite high. One last comment might be even the highest level model, the reproducibility of the experimental results is not perfect. There could be some deficiencies in the experiment but also since the initial and boundary conditions always have uncertainty, you may need always kind of sensiticvity analysis or ensemble simulations. At any rate, I think the paper is acceptable for the publication.
Citation: https://doi.org/10.5194/egusphere-2024-2588-RC3 -
AC2: 'Reply on RC3', Zhi Li, 02 Dec 2024
Thank you for the thoughtful review and positive feedback regarding our manuscript on the SERGHEI-RE model. We appreciate your recognition of the clarity of our content and the high-performance computing aspects of our work.
We acknowledge your concern regarding the reproducibility of experimental results, particularly in light of uncertainties associated with initial and boundary conditions. Many of the available benchmark problems in the literature involves surface-subsurface exchange and/or solute transport. The SERGHEI model is also being developed towards these directions, but this manuscript only focuses on the subsurface flow. We found limited number of benchmark test problems that came from real-world experiments, considered subsurface flow only, and provided high-quality data for the initial and boundary conditions. However, we are preparing another manuscript to demonstrate the capability of SERGHEI in simulating surface-subsurface flow exchange. There will be more persuasive demonstrations in the upcoming manuscript.
Thank you once again for your constructive comments.
- Zhi Li
Citation: https://doi.org/10.5194/egusphere-2024-2588-AC2
-
AC2: 'Reply on RC3', Zhi Li, 02 Dec 2024
Status: closed
-
RC1: 'Comment on egusphere-2024-2588', Anonymous Referee #1, 01 Oct 2024
In the manuscript titled “SERGHEI v2.0: introducing a performance-portable, high-performance three-dimensional variably-saturated subsurface flow solver (SERGHEI-RE)”, the authors develop the Richards Equation-based variably-saturated subsurface flow module of SERGHEI. The characteristics of the framework are accuracy, robustness, scalability, and portability, which preserves the parallel performance and scalability across different HPC hardware. There are still certain changes and clarifications that the authors should address prior to publication. For these reasons, I believe that the manuscript can be accepted for publication by the GMD after minor revision. Below, I have some general comments for the authors.
General comments:
- Line #48 – 49, it would be better provide further explanation on how parallel performance depends on HPC hardware. For example, are the differences in the parallel computing scripts caused by the different HPC compilers, scheduler, or are the differences due to the storage methods (e.g. HPSS, GPFS file system) used by the HPC? This would help readers quickly understand the practical problems that this study aims to solve. If only the compilation issues related to CPU parallelization and GPU parallelization are addressed (line #59-60), please appropriately discuss the limitations of this study.
- Line #124-129, it would be necessary to provide specific examples of the use of PC and MP schemes in SERGHEI-RE. This would make it more intuitive and help better understand the advanced nature of this research plan.
- Equation 8 could be illustrated with a diagram like Fig.2.
- The comparison figures of PC and MP show that, except for the 4 days simulation in Fig. 4, they almost completely overlap. The introduction section regarding these two different types of introductions cannot fully explain the outcome. Therefore, it is best to provide further explanation.
- Figure 11, almost all evaporation points are zero after 50 days, this seems to be caused by a model code error. If not, please explain briefly.
Specific comments:
- Figure 1, in this diagram, what is the difference between the dashed lines, solid lines, and those without borders?
- Line #100, what do α and represent?
Citation: https://doi.org/10.5194/egusphere-2024-2588-RC1 -
AC1: 'Reply on RC1', Zhi Li, 23 Oct 2024
We thank the reviewer for providing constructive comments on our work. Please find our response in the attachments.
Zhi Li
-
RC2: 'Reply on AC1', Anonymous Referee #1, 31 Oct 2024
I am sorry for the misunderstanding, the "script" in the first comment means the workflows to build the CPU/GPU binaries.
The reviewers have diligently addressed all the comments and revisions I provided during the previous review.Citation: https://doi.org/10.5194/egusphere-2024-2588-RC2
-
RC2: 'Reply on AC1', Anonymous Referee #1, 31 Oct 2024
-
RC3: 'Comment on egusphere-2024-2588', Anonymous Referee #2, 01 Dec 2024
The paper presents the development of a high-performance three-dimensional variably-saturated subsurface flow slover (SERGEI-RE). The contents is clear and shows good results. Likewise, the level of high performance computing is quite high. One last comment might be even the highest level model, the reproducibility of the experimental results is not perfect. There could be some deficiencies in the experiment but also since the initial and boundary conditions always have uncertainty, you may need always kind of sensiticvity analysis or ensemble simulations. At any rate, I think the paper is acceptable for the publication.
Citation: https://doi.org/10.5194/egusphere-2024-2588-RC3 -
AC2: 'Reply on RC3', Zhi Li, 02 Dec 2024
Thank you for the thoughtful review and positive feedback regarding our manuscript on the SERGHEI-RE model. We appreciate your recognition of the clarity of our content and the high-performance computing aspects of our work.
We acknowledge your concern regarding the reproducibility of experimental results, particularly in light of uncertainties associated with initial and boundary conditions. Many of the available benchmark problems in the literature involves surface-subsurface exchange and/or solute transport. The SERGHEI model is also being developed towards these directions, but this manuscript only focuses on the subsurface flow. We found limited number of benchmark test problems that came from real-world experiments, considered subsurface flow only, and provided high-quality data for the initial and boundary conditions. However, we are preparing another manuscript to demonstrate the capability of SERGHEI in simulating surface-subsurface flow exchange. There will be more persuasive demonstrations in the upcoming manuscript.
Thank you once again for your constructive comments.
- Zhi Li
Citation: https://doi.org/10.5194/egusphere-2024-2588-AC2
-
AC2: 'Reply on RC3', Zhi Li, 02 Dec 2024
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 236 | 58 | 138 | 432 | 2 | 5 |
- HTML: 236
- PDF: 58
- XML: 138
- Total: 432
- BibTeX: 2
- EndNote: 5
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1