the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
New and existing covariance estimation techniques for ensemble data assimilation
Abstract. Covariance estimation from a small number of samples is a challenging, but necessary task in ensemble data assimilation (DA) and a fundamental problem in statistics and machine learning. Many covariance estimation methods have been developed in geophysics and in statistics, with little exchange of ideas between the two fields. Covariance estimation in statistics and in ensemble DA rely on fundamentally different assumptions, and we study the efficiency and applicability of both approaches to covariance estimation in systematic numerical experiments. Our numerical experiments are designed to feature a correlation structure in which correlation decays with distance globally, but the rate of decay is location-dependent. In such cases, our findings suggest that methods that make even minimal assumptions on the decay of spatial correlations are more accurate than those that do not do so at all. We also describe how to use a generalized Gaspari-Cohn correlation function to design new covariance estimation methods that capture intricate and spatially varying correlation structures and thereby further reduce covariance estimation errors.
- Preprint
(6838 KB) - Metadata XML
- BibTeX
- EndNote
Status: final response (author comments only)
- RC1: 'Comment on egusphere-2026-5032', Anonymous Referee #1, 21 Oct 2026
-
RC2: 'Comment on egusphere-2026-5032', Anonymous Referee #2, 04 Nov 2026
The manuscript compares covariance estimators for small-ensemble DA (GC and GenGC localization, hybrid estimators, Ledoit–Wolf shrinkage, soft thresholding and NICE) on a static Gaussian problem and in cycling EnKF experiments with two inhomogeneous Lorenz-96 variants, and introduces GenGC localization with spatially varying cut-off lengths. The experiments are carefully executed and the paper is well written.
My main concern is scope. The paper is framed as a bridge between covariance estimation in statistics and in ensemble DA, but the family of structured precision-matrix estimators (banded/modified Cholesky, graphical lasso, and their EnKF implementations) is absent from both the review and the experiments, although Bickel and Levina (2008b) and Friedman et al. (2008) are cited. These estimators are SPD by construction and encode locality through conditional independence rather than tapering, so they address the two problems the paper itself documents (non-PSD estimates and the lack of a notion of distance). Their omission makes the conclusion that statistical estimators are not competitive with localization premature.
A second concern is that the GenGC cut-off function is prescribed from the model’s known forcing or damping profile rather than estimated from data, so its practical applicability is unclear, and the gain over standard GC is marginal.
Detailed comments, including on the tuning protocol, the single ensemble size, terminology and missing experimental details, are given in the attached supplement. I recommend major revisions.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 337 | 184 | 66 | 587 | 61 | 62 |
- HTML: 337
- PDF: 184
- XML: 66
- Total: 587
- BibTeX: 61
- EndNote: 62
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1
Please see attached pdf for review