Covariance estimation from a small number of samples is a challenging, but necessary task in ensemble data assimilation (DA) and a fundamental problem in statistics and machine learning. Many covariance estimation methods have been developed in geophysics and in statistics, with little exchange of ideas between the two fields. Covariance estimation in statistics and in ensemble DA rely on fundamentally different assumptions, and we study the efficiency and applicability of both approaches to covariance estimation in systematic numerical experiments. Our numerical experiments are designed to feature a correlation structure in which correlation decays with distance globally, but the rate of decay is location-dependent.  In such cases, our findings suggest that methods that make even minimal assumptions on the decay of spatial correlations are more accurate than those that do not do so at all.  We also describe how to use a generalized Gaspari-Cohn correlation function to design new covariance estimation methods that capture intricate and spatially varying correlation structures and thereby further reduce covariance estimation errors.