the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 14 Nov 2025
Improvement of the Computational Efficiency in SVD-3DEnVar Data Assimilation Scheme and Its Preliminary Application to the TRAMS 3.0 Model
Abstract. Although the Singular Value Decomposition-three Dimensional Ensemble Variational (SVD-3DEnVar) data assimilation scheme has achieved successful application in real case simulations with comprehensive numerical weather prediction models, its computational efficiency still cannot meet the demands of actual operational numerical forecasting. The main limitations lie in the generation of three-dimensional perturbations and the implementation of parallel calculations. This paper constructed a three-dimensional perturbation field generation scheme that supports multi-process parallelism and can directly generate any specified number of grid points in both horizontal and vertical directions. At the same time, an efficient parallel implementation scheme has been developed according to the characteristics of local patch assimilation in the SVD-3DEnVar scheme. The Observing System Simulation Experiment (OSSE) test results based on the Tropical Regional Atmospheric Model System (TRAMS) show that after computational efficiency optimization, the time required to generate a 3D perturbation field has been reduced from 22 minutes to 2.2 seconds, while the runtime of the assimilation process has decreased from 1,700 minutes under serial execution to less than 15 minutes (using 150 nodes in parallel). Finally, we conducted an assimilation experiment using actual observational data of sea surface wind fields to preliminarily validate the reasonableness of the assimilation results from the optimized SVD-3DEnVar scheme.
- Preprint
(2020 KB) - Metadata XML
- BibTeX
- EndNote
Status: open (until 09 Jan 2026)
-
RC1: 'Comment on egusphere-2025-4632', Anonymous Referee #1, 17 Nov 2025
reply
-
AC1: 'Reply on RC1', Kun Liu, 22 Nov 2025
reply
Response to Reviewer 1
We sincerely thank the reviewer for the insightful comments and constructive suggestions, which have significantly helped us improve the quality of our manuscript. We have carefully considered all points and have revised the manuscript accordingly. Below, we provide a point-by-point response to the comments.
Comment 1:
Regarding the choice of methodology, it is recommended that you explain why SVD-4DVar was not adopted in favor of SVD-3DVar, while briefly analyzing the core challenges of the latter.
Response:
We appreciate the reviewer’s suggestion. The decision to adopt SVD-3DEnVar instead of SVD-4DEnVar was primarily motivated by two factors:
Computational Efficiency and Operational Feasibility: The main objective of this study is to improve computational efficiency for operational typhoon forecasting, so the relatively simple 3DVar scheme is chosen for experimentation...
Observation Frequency: The sea surface wind observations used in our real-data experiments are available only every 6 hours. This low temporal resolution does not fully leverage the temporal continuity advantages of 4DEnVar.
We acknowledge that SVD-3DEnVar has its own challenges, which we now discuss in the revised manuscript (Section 5). The primary limitations include: The use of a single localization scale, which may not optimally handle multi-scale observations (e.g., surface, satellite, and radar); The limited ability of linear combinations of singular vectors to represent strongly nonlinear relationships, especially for unobserved variables. These challenges will guide our future work on multi-scale assimilation and machine-learning-enhanced methods.
Comment 2:
It should be noted that the singular value decomposition (SVD) of matrix A is extremely challenging in practice due to its large dimensions (Nx + Ny), posing significant difficulties in terms of both storage and computation. A discussion on this aspect is recommended.
Response:
We fully agree with the reviewer. To address the computational and storage challenges of performing SVD on the large matrix A, we implemented a local patch assimilation strategy. This approach significantly reduces the effective dimensions of A by horizontal and vertical localization. Only observations within a specified horizontal (lh) and vertical (lv) radius from the central grid point are included. As a result, both Nx (model variables in the local patch) and Ny (observations within the local patch) are drastically reduced. This makes the SVD computationally feasible without sacrificing the flow-dependent covariance information. We have added a clarification in Section 3.1 to explain this strategy.
Comment 3:
While equation (11) provides the Gaussian weight function, the localization scheme used in SVD-3DVar should be presented in more detail to enhance the completeness of the paper.
Response:
Thank you for this suggestion. We have expanded the description of the localization scheme in Section 3.1. The revised text now reads:
“The Gaussian weight function defined in Equation (11) is applied to each observation within the local patch. The horizontal and vertical localization scales (σh and σv) control the rate at which observation influence decays with distance. This localization ensures that only observations within a specified radius significantly impact the analysis increment at the center of the local patch, thereby mitigating spurious long-range correlations and improving the stability and accuracy of the assimilation.”
Comment 4:
In recent years, 4DEnVar methods have advanced rapidly. To reflect an up-to-date understanding of the field, it is advisable to include references to relevant studies published between 2022 and 2025.
Response:
We thank the reviewer for this suggestion. We have now incorporated several recent references on 4DEnVar advancements in the Introduction and Conclusion sections, including:
Inverarity et al. (2023) on hybrid En-4DEnVar in the Met Office system;
Berre and Arbogast (2024) on hybrid covariances at Météo-France;
Lu and Wang (2024) on scale-dependent localization in hurricane forecasting;
Thiruvengadam and Wang (2025) on convective-scale 4DEnVar;
Wang et al. (2025) on CubeSat radiance assimilation.
These additions help contextualize our work within the evolving landscape of ensemble-variational methods.
Comment 5:
Regarding the generation of initial samples, several classical works (e.g. those by Evensen) have achieved high memory efficiency. It would be beneficial to reference these works and discuss their relevance to the present method. From a practical perspective, the main computational burden in parallelization typically lies in the ensemble forecast component, which should also be addressed.
Response:
We agree with the reviewer. The initial perturbation generation in SVD-3DEnVar is based on the Gaussian random field method introduced by Evensen (1994), which is known for its memory efficiency and statistical robustness. However, the original implementation was limited to 2D square grids with odd numbers of points, which motivated our multi-dimensional and parallel optimizations. We have mentioned Evensen’s foundational work and clarify how our optimizations build upon it.
Regarding the computational burden of ensemble forecasts: yes, the ensemble integration is indeed the most computationally intensive part in parallel implementations. However, since ensemble members are independent, they can be run concurrently if sufficient computational resources are available. While this study focuses on optimizing the perturbation generation and assimilation steps, we acknowledge the resource demands of ensemble forecasting and will address this in future work.
Comment 6:
As this is an ensemble-based method, it is recommended that the ensemble sample update strategy in SVD-3DVar is explained briefly to improve the completeness of the methodological description.
Response:
We have added the following explanation to Section 1:
“Unlike traditional ensemble assimilation schemes (such as EnKF), which directly update each ensemble member during cyclic assimilation, SVD-3DEnVar only assimilates and updates the control forecast. Therefore, after each assimilation cycle, the updated analysis field must be perturbed again to generate the initial conditions for the next cycle's ensemble forecast. This approach has the advantage of avoiding filter divergence and, in the presence of model errors, performs better than EnKF (Qiu et al., 2007).”
-
AC1: 'Reply on RC1', Kun Liu, 22 Nov 2025
reply
-
CC1: 'Comment on egusphere-2025-4632', Nima Zafarmomen, 03 Dec 2025
reply
The paper targets computational bottlenecks in the SVD-3DEnVar data assimilation scheme and proposes two main engineering improvements: (i) a redesigned perturbation-field generation algorithm that directly produces 3D, grid-conforming perturbations with multi-process parallelism and without intermediate I/O, and (ii) a parallel local-patch assimilation framework that partitions work, balances load, and reduces memory duplication via node-level pointer sharing. Using the TRAMS 3.0 model, the authors report large wall-clock reductions: 3D perturbation generation from ~22 minutes to 2.2 seconds and end-to-end assimilation from ~1700 minutes (serial) to <15 minutes with 150 nodes, alongside preliminary OSSE and real-data tests that show reasonable analysis increments and some improvement in typhoon forecasts.
Overall, the paper tackles a very practical problem: making SVD-3DEnVar fast and memory-efficient enough for realistic, large-domain applications. I. recommed it for publication after considering these comments:
- How do you prescribe and verify the target covariance statistics of the perturbation fields (variance, horizontal and vertical correlation lengths, cross-variable correlations)?
- Are perturbations generated multivariately (i.e., with controlled cross-variable balance), or independently per variable with post hoc smoothing?
- What are the precise observation operator and error model used for the 10 m sea-surface winds (e.g., stability-dependent surface-layer mapping, bias correction, representativeness error)? How are coastal/land points handled, and are rainy scenes screened?
- Please clarify the SVD/variational notation: define Λp explicitly, correct the SVD equation symbols, and detail the truncation criterion for K (energy, cross-validated error, or fixed)?
- I storngly recommend to expand your introduction and cite below paper
Assimilation of sentinel‐based leaf area index for modeling surface‐ground water interactions in irrigation districts
-
Abstract:
-
Consider explicitly stating that the main novelty is the computational optimization and parallelization that makes SVD-3DEnVar suitable for operational use, plus a first real-data test with satellite-derived sea surface winds.
-
-
Notation consistency:
-
Ensure all symbols in the equations are defined once and consistently (e.g., Λ vs Λ_P in Equation (10); clarify if Λ_P is the same eigenvalue matrix as in Eq. (4) or modified).
-
Make sure the dimension of vectors and matrices in Eqs. (1)–(10) is always clear (model vs observation subspaces).
-
-
Equation (11) Gaussian localization:
-
Check the condition in the piecewise definition:
-
You currently use (r_h ≤ l_h and r_v ≤ l_v) vs (r_h > l_h or r_v ≥ l_v). It might be clearer and more symmetric to use strict/≤ consistently.
-
-
Consider giving an example of actual physical localization scales (in km) corresponding to l_h, l_v, σ_h, σ_v.
-
Citation: https://doi.org/10.5194/egusphere-2025-4632-CC1
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 144 | 27 | 18 | 189 | 11 | 11 |
- HTML: 144
- PDF: 27
- XML: 18
- Total: 189
- BibTeX: 11
- EndNote: 11
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
The SVD-3/4DVar method, proposed by Qiu et al. (2007), is considered a pioneering achievement in the field of four-dimensional ensemble variational data assimilation (4DEnVar). This manuscript provides a valuable exploration of the practical application of the SVD-3DVar method and demonstrates certain innovative merits, meeting the publication criteria of this journal. The following suggestions are provided for further improvement:
1. Regarding the choice of methodology, it is recommended that you explain why SVD-4DVar was not adopted in favour of SVD-3DVar, while briefly analysing the core challenges of the latter.
2. It should be noted that the singular value decomposition (SVD) of matrix A is extremely challenging in practice due to its large dimensions (Nx+Ny), posing significant difficulties in terms of both storage and computation. A discussion on this aspect is recommended.
3. While equation (11) provides the Gaussian weight function, the localization scheme used in SVD-3DVar should be presented in more detail to enhance the completeness of the paper.
4. In recent years, 4DEnVar methods have advanced rapidly. To reflect an up-to-date understanding of the field, it is advisable to include references to relevant studies published between 2022 and 2025.
5. Regarding the generation of initial samples, several classical works (e.g. those by Evensen) have achieved high memory efficiency. It would be beneficial to reference these works and discuss their relevance to the present method. From a practical perspective, the main computational burden in parallelisation typically lies in the ensemble forecast component, which should also be addressed.
6. As this is an ensemble-based method, it is recommended that the ensemble sample update strategy in SVD-3DVar is explained briefly to improve the completeness of the methodological description.