the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 27 Sep 2024
Enhancing Single-Precision with Quasi Double-Precision: Achieving Double-Precision Accuracy in the Model for Prediction Across Scales-Atmosphere (MPAS-A) version 8.2.1
Abstract. The development of numerical models are constrained by the limitations of high performance computing (HPC). Low precision computations can significantly reduce computational costs, but inevitably introduce rounding errors, which affect computational accuracy. Quasi double-precision algorithm can compensate for rounding errors by keeping corrections, thereby achieving the low numerical precision while maintaining result accuracy. This paper applies the algorithm to the Model for Prediction Across Scales-Atmosphere (MPAS-A) and evaluate its performance across four test cases. The results demonstrate that, after reducing numerical precision to single precision (from 64 bits to 32 bits), the application of quasi double-precision algorithm can achieve results comparable to double-precision computations. The round-off error of surface pressure is reduced by 68 %, 75 %, 97 %, 96 % in cases, the memory has been reduced by almost half, while the computation increases only 2 %, significantly reducing computational cost. The work substantiates both effectiveness and inexpensive computation in numerical models by using quasi double-precision algorithm.
- Preprint
(3054 KB) - Metadata XML
-
Supplement
(393 KB) - BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on egusphere-2024-2986', Anonymous Referee #1, 04 Oct 2024
In the manuscript titled “Enhancing Single-Precision with Quasi Double-Precision: Achieving Double-Precision Accuracy in the Model for Prediction Across Scales-Atmosphere (MPAS-A) version 8.2.1”, the authors develop quasi double-precision algorithm, which can compensate for rounding errors by keeping corrections, thereby achieving the low numerical precision while maintaining result accuracy. By evaluating Model for Prediction Across Scales-Atmosphere (MPAS-A) performance, the algorithm can achieve results comparable to double-precision computations. There are still certain changes and clarifications that the authors should address prior to publication. For these reasons, I believe that the manuscript can be accepted for publication by the GMD after major revision. Below, I have some comments for the authors.
Major comments:
- Based on Figure 1 and Figure 2, this computational method appears somewhat simple. There is relatively little research on the keyword "quasi double-precision." The Authors could explain why their work is novel compared to the existed methods already published nowadays.
- The main point of applying signal precision computing methods is ensuring predictive, and reducing computational costs. The iterative precision compensation increases the computation load. Has this study considered the issue of computational efficiency? For example, runtime, reduced computational cost, they were mentioned in introduction literature review, but not studied in this study.
- The model is described poorly. The solution method for the equations is not even mentioned. For example, the finite difference scheme is mainly used to calculate the primary equations for variables studied in this work. And at which step of the equation is the quasi double-precision algorithm specifically applied? The strategy used to compute cell edge, dry air density, potential temperature with quasi double-precision algorithm is also difficult to understand from Figure 3.
- The color scheme in Figure 5 is very hard to distinguish; the solid purple area is too large, making the gradient difficult to see. Figure 10 has the same issue. It is recommended to refer to the classic NCL color scheme. https://www.ncl.ucar.edu/Applications/era40.shtml
- Figure 6 (a) shows that DBL-SGL decreases after 1.0, which appears to be caused by a coding error. Please check and confirm the validity of the data.
Minor comments:
- The color bar in Figure 7 seems almost useless.
- Please mention “round-off error” in relevant figures' captions.
- Figure 11 appears to be somewhat blurry.
Citation: https://doi.org/10.5194/egusphere-2024-2986-RC1 -
AC1: 'Reply on RC1', Jiayi Lai, 17 Oct 2024
We sincerely appreciate your constructive comments and insightful suggestions on our manuscript. Your feedback has been invaluable in refining our work and improving the analysis. We have prepared a general response, along with a point-by-point response to your comments, which is included in the attached document.
We remain open to any further suggestions or discussions that could help enhance our work.
-
RC2: 'Reply on AC1', Anonymous Referee #1, 18 Oct 2024
Response to Point 1:
The calculation method used in this paper is based on the method of Møller, O. (1965)., which appears relatively simple. If the authors can find similar studies that employ this "quasi double-precision" method for simulating atmospheric variables, it could demonstrate the advancement of this paper compared to existing research. Chen et al. (2024) and Vánˇa et al. (2016) did not use this method. If no similar studies are found, it could be stated that this paper is the first to do so, which could enhance its citation rate.Response to Point 3:
Regarding equations (6)-(9), it is suggested to select only one equation and present it in the form of Figure 2. This could be included as an appendix to specifically demonstrate the iterative calculation process. Because the pseudo-code in Fig. 3 is not sufficient to clearly explain the calculation process.Response to Point 5:
If the authors intend to replace the existing Figures 4, 6, and 8 (in the manuscript) with these updated versions (Figures 4, 5, and 6), feedback from other editors and reviewers regarding the revised structure of the paper should be considered.The new figures:
There are several cases where figure legends and lines overlap, such as in Figure 3 in the review response.
Reference:
Møller, O. (1965). Quasi double-precision in floating point addition. BIT Numerical Mathematics, 5(1), 37-50.
Citation: https://doi.org/10.5194/egusphere-2024-2986-RC2 - AC2: 'Reply on RC2', Jiayi Lai, 18 Oct 2024
-
RC2: 'Reply on AC1', Anonymous Referee #1, 18 Oct 2024
-
RC3: 'Comment on egusphere-2024-2986', Anonymous Referee #2, 21 Oct 2024
The manuscript titled “Enhancing Single-Precision with Quasi Double-Precision: Achieving Double-Precision Accuracy in the Model for Prediction Across Scales-Atmosphere (MPAS-A) version 8.2.1” explores the use of the quasi-double precision algorithm by Møller et al. (1965) within a numerical modelling framework. By implementing the algorithm in MPAS-A across various case studies and idealized scenarios, the study highlights its potential to lower computational costs compared to double-precision approaches while enhancing accuracy over single-precision approaches.
Overall, the manuscript is well-organized and relatively clear, but there are several areas that could benefit from further clarification and elaboration. In particular, I have concerns regarding 1) the practical significance of the reported improvements, 2) the lack of detail and rationale in the model setup, and 3) the insufficient explanation or justification for the choice of metrics. Based on these issues, I recommend major revisions before the manuscript can be reconsidered for publication.
Below, you find a list of my main concerns:
1. The practical impact of the improvements achieved with the quasi-double precision algorithm compared to the single-precision approach remains unclear to me. While the authors demonstrate large relative accuracy gains compared to single-precision, it is not evident that these improvements are meaningful in absolute terms. For example, in the idealized scenario of Section 3.1, the RMSE of surface pressure improves by only 10^-2 Pa at longer lead times (12+ days), with negligible improvements at shorter times. Similar patterns emerge in Section 3.3. Given that these improvements are an order of magnitude smaller than the World Meteorological Organization's (WMO) recommended measurement uncertainty for atmospheric pressure (e.g., https://gcos.wmo.int/en/essential-climate-variables/pressure), it might be difficult to justify the additional computational costs. I encourage the authors to elaborate on the significance of their results in absolute terms from both a practical and forecasting perspective.
A related point: Are current operational implementations of MPAS-A defaulting to double- or single-precision for the variables discussed in this manuscript? Given the results, if double precision is currently the standard, the authors might want to emphasise the potential of quasi-double precision as a substitute for double precision, rather than focusing primarily on its benefits over single-precision.
2. The experimental setup lacks sufficient detail and justification. It is unclear why the authors chose these particular case studies and idealised scenarios, especially in the applications to real data. This makes it difficult to assess the generalisability of the results to other atmospheric conditions or variables. Additionally, a more detailed comparison of computational costs among the three algorithms (single-precision, quasi-double precision, and double precision) is necessary, as improvements in accuracy may not be feasible in practice if they come with substantial increases in computational expense.
3. The manuscript currently employs only grid-point level and "spatial RMSE" as comparison metrics, but the term "spatial RMSE" is not clearly defined. I recommend the following:
1) Clearly define "spatial RMSE," particularly when used as a summary statistic over a larger domain. For example, are the authors applying any latitude-based weighting?
2) Justify the use of this metric and consider comparing the algorithms using additional relevant metrics. If alternative metrics are not applicable, please explain why they were not used.Minor concerns/technical corrections:
- Figure 6a: The higher error with the quasi-double precision algorithm compared to the single-precision in this subpanel is confusing. If the error with the single-precision algorithm is anyway so small that it does not matter in practice, why was this case study selected, and why include this figure?
- Figure 7: The phrase "the circle represents the most clear error" lacks objectivity, especially since other areas in the figure show similar errors. Consider rephrasing or finding a more precise way to characterize the comparison between errors in this case.
- Please avoid using the term "significant" or "significantly" when not referring to statistically significant results to avoid potential confusion.
- Unclear phrase (line 95): "achieves basically consistent results comparable to those of double precision" – Do you mean that the accuracy is similar in a specific case, or that the quasi-double precision consistently performs similarly to double precision in most cases?
- Reproducibility note: Although the manuscript adheres to GMD's guidelines by providing a DOI to a permanent repository, no instructions are provided on how to use the code or what the various files and subfolders contain. I suggest including at least a brief README file that describes the content and organization of the repository, providing users with clearer guidance.
Citation: https://doi.org/10.5194/egusphere-2024-2986-RC3 -
AC3: 'Reply on RC3', Jiayi Lai, 31 Oct 2024
We sincerely appreciate your constructive comments and insightful suggestions on our manuscript. Your feedback has been invaluable in refining our work and improving the analysis. We have prepared a general response, along with a point-by-point response to your comments, which is included in the attached document.
We remain open to any further suggestions or discussions that could help enhance our work.
-
AC3: 'Reply on RC3', Jiayi Lai, 31 Oct 2024
-
RC4: 'Comment on egusphere-2024-2986', Anonymous Referee #3, 30 Oct 2024
In their manuscript "Enhancing Single-Precision with Quasi Double-Precision: Achieving Double-Precision Accuracy in the Model for Prediction Across Scales-Atmosphere (MPAS-A) version 8.2.1," the authors introduce a quasi double-precision (QDP) correction approach, specifically using the method proposed by Møller (1965). This method addresses precision loss by maintaining correction terms to reduce rounding errors, especially when adding small values to larger ones in floating-point arithmetic. The QDP method is implemented in the time integration steps of the model’s dynamical core, where it compensates for rounding errors that arise in single-precision calculations. Its effectiveness is evaluated across four test scenarios, including idealised and real-world applications. The authors’ findings suggest that the QDP correction significantly reduces rounding errors, yielding results closer to double-precision computations.
While the study presents promising findings, certain aspects require clarification. The computational impact is mentioned in the abstract and conclusions but is not substantiated or elaborated elsewhere in the manuscript. Additionally, the primary contribution—the QDP correction’s implementation within MPAS-A—is presented in a brief section, with only about 10 lines describing the process. This limited description does not fully convey the extent and specifics of the adjustments made, making it hard to gauge the required level of effort or intricacies of the modification. Furthermore, the experimental framework lacks an uncertainty analysis or a comparison to alternative sources of error, which would strengthen the credibility of the findings.
For these reasons, I recommend that the manuscript undergo major revisions to address these points before it is considered for publication in Geoscientific Model Development (GMD).
Major Comments
- The abstract states: “Low precision computations can significantly reduce computational costs, but inevitably introduce rounding errors, which affect computational accuracy.” However, it is not clearly defined what is meant by “low precision computations,” and the statement that such computations inevitably affect accuracy may not always hold true.
- Both the abstract and conclusions mention the computational impact of the proposed methods; however, these effects are not discussed further within the main body of the manuscript.
- Additional context would be beneficial in distinguishing which differences are relevant. Including an uncertainty analysis would help place the magnitude of the errors into perspective. For example, Figure 8 shows significant differences in errors between low- and high-resolution grids, with these discrepancies appearing more impactful than those arising from precision changes alone.
- Section 2.3 would benefit from further detail on which parts of the code were modified and how these sections were selected for modification.
- In Section 3.1, the differences between cases only emerge after 10 days of integration. It would be valuable to contextualize these differences with the error growth from other potential sources of uncertainty.
- Section 3.2 states that “the errors are very small and can be ignored.” More context is needed here to help determine which differences are meaningful.
- In Section 3.3, the authors note that “Differences in error begin to emerge after 500 steps.” This could be strengthened by comparing this error growth to that of other sources of uncertainty, some of which may become relevant earlier in the integration process.
Minor Comments
- Line 19 – The authors reference a 2015 source to indicate that systems are expected to grow. While this is still valid, the “current systems” referenced in 2015 are no longer today’s current systems.
- Line 47 – The mixed-precision reference for NEMO notes that “95.8% of the 962 variables could be computed using half precision,” though the publication itself refers to single precision.
- Line 143 – It is unclear if this version of MPAS-A is indeed the only one capable of utilizing single-precision.
- Figures 5, 7, 9, 10, 11, and 12 – Alternative colormaps are suggested for all figures containing maps.
- Line 241–242 – The phrase “but the process is more sensitive for the precision” lacks clarity and would benefit from rephrasing.
Citation: https://doi.org/10.5194/egusphere-2024-2986-RC4 -
AC4: 'Reply on RC4', Jiayi Lai, 05 Nov 2024
We sincerely appreciate your constructive comments and insightful suggestions on our manuscript. Your feedback has been invaluable in refining our work and improving the analysis. We have prepared a general response, along with a point-by-point response to your comments, which is included in the attached document.
We remain open to any further suggestions or discussions that could help enhance our work.
-
RC5: 'Comment on egusphere-2024-2986', Anonymous Referee #4, 31 Oct 2024
"Enhancing Single-Precision with Quasi Double-Precision: Achieving Double-Precision Accuracy in the Model for Prediction Across Scales-Atmosphere (MPAS-A) version 8.2.1" presents a quasi double precision method implemented in MPAS, which reduces memory costs with the trade off of only slightly increasing computational costs. The method relies on applying correction factors to the single precision variables when solving. The application of quasi double-precision achieves results more comparable to double-precision computations. The authors present both idealized and real data test cases to analyze the error in properties such as surface pressure, 500 hPa heights, and kinetic energy to illustrate how the method performs in terms of reducing error.
While this work could have interesting implications when applied to other variables in the model, it currently addresses only a handful of variables. However this is hopefully an encouraging first step. That said, the manuscript requires improvements to clarify what was done and help frame its implications. Improving the manuscript is important as this work can be seen as useful for future work as it was only applied and tested for a small portion of the model (the dynamical core variables) and not applied model-wide to tracer variables.
Specific comments:
- The algorithm description needs more attention and description in the text. By comparison, the description of MPAS seems much better detailed and it could be argued that much of it is less important. I recommend finding ways to enhance the content of section 2.1.
- For clarification, is the QDP runtime is 2% more expensive compared to the DBL runtime or the SGL runtime? Also how much reduction in runtime was there for the SGL version compared to the DBL version? This can be used to give context to what benefits are achieved for the more noticeable errors in SGL.
- Given errors appear to become noticeably larger beyond 10+ days, how are these figures dependent with different spatial time average lengths? Do they all only look drastically different because of errors after round off error becomes significant around day 10+? Is this error tied to simulation length or is this more determined by the number of time steps taken?
- In some experiments, the reduction of error is a large percentage but feels like a small error. In general to the reader, it's not very clear that reducing these errors will matter that much. For example, while the error is reduced using QDP in a lot of cases, reduction of the surface pressure (hPa) error by a RSME of 2.8x10-1 to 1.4x10-1 seems like a small amount. Likewise for other sources of error and that this all needs context. It is important to consider these errors relative to other sources of model errors.
- Additionally, I was wondering if the authors could leverage the test case where grid resolution was also incorporated as the model discretization is a source of model error compared to the model error due to precision differences.
- For the cases in 3.3, it is not clear why the decision to have the higher resolution simulation of 120 km x 120 km have the same time step of 720 seconds as the 240 km x 240 km case. This seems rather unusual as typically a reduction in grid cell size is accompanied by a proportionate reduction in time step size to maintain a consistent Courant number.
Minor comments:
- Figure 1, Figure 2 and a lesser extent Figure 3: The algorithm are written in a way that is difficult to read. I suggest that it is better distinguished the text (i.e. "evaluation") from variables (i.e. s,u,v). In Figure 2, everything is appears quite mathematical while Figure 3 is quite the opposite. It would be beneficial to have these all looking the same.
- Line 150: "so we close the scalar transport in all cases". Does this mean it's turned off/not solved? Unless "closed" is a common term in the MPAS community, I suggest the authors consider a term closer to "disable/disabled" instead of "close/closed" when talking about model processes that are not absent in the simulation.
- Line 179: Has this set of simulation parameters been utilized in a previous published work as a test case? This would be preferred to stating it was on a website that is subject to change.
- Line 195 states the resolution is 84 km x 84 km where I believe this is actually the total domain size and not resolution. Assuming my understanding is correct, how many grid cells are within this domain and/or what is the average grid cell size in this simulation? Is it the ~500 m in Klemp 2015?
- Line 197: While the errors in Figure 6(a) are indeed quite small, it may be beneficial to say how small relative to the actual values of kinetic energy? And I think a more clear statement explaining this figure's behavior would help clarify the plot as it's not the expected behavior throughout the manuscript that QDP does better in terms of error.
- Figure 7: The location of the Y (km) should be moved up near the axes (and should probably be labeled X?) and not under the color bar and Z is used which is typically thought of as a vertical direction is on the y axis. Color bar should probably have different scaling with a smaller range. Additionally why are the circles different colors between the different panels?
- Figure 9 is one of the only figures that has color bar limits change between the SGL and QDP panels. Given that quite often the color bar limit have been fixed ranges in most figures, a note in the caption may be beneficial for this.
- Figure 10(b) uses the same color bar as Figure 10(a) but reveals no detail because the same color bar limits are too large. I suggest that a new color bar range is selected (and it is then noted in the caption as being different in an and b as suggested for Figure 9).
- For code and data availability section, it is mentioned that "model code and plotting data" is available While I was able to find the versions of the model code, the simulation inputs and plotting scripts, I was unable to locate actual data. It's unclear if this is intended as the wording indicates "plotting data" and we should rather anticipate running the model to produce the data.
Technical comments:
- Negative exponents have been routinely miswritten throughout the manuscript such as 10-2 rather than 10-2.
- Line 218: "The Spatial RMSE of with 120 km x 120km.." is missing the variable name, which appears to be surface pressure.
Citation: https://doi.org/10.5194/egusphere-2024-2986-RC5 -
AC5: 'Reply on RC5', Jiayi Lai, 05 Nov 2024
We sincerely appreciate your constructive comments and insightful suggestions on our manuscript. Your feedback has been invaluable in refining our work and improving the analysis. We have prepared a general response, along with a point-by-point response to your comments, which is included in the attached document.
We remain open to any further suggestions or discussions that could help enhance our work.
-
AC5: 'Reply on RC5', Jiayi Lai, 05 Nov 2024
- AC6: 'Comment on egusphere-2024-2986', Jiayi Lai, 10 Dec 2024
- AC7: 'Comment on egusphere-2024-2986', Jiayi Lai, 10 Dec 2024
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 314 | 80 | 191 | 585 | 23 | 5 | 7 |
- HTML: 314
- PDF: 80
- XML: 191
- Total: 585
- Supplement: 23
- BibTeX: 5
- EndNote: 7
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1