Abstract:Modern datasets are trending towards ever higher dimension. In response, recent theoretical studies of covariance estimation often assume the proportional-growth asymptotic framework, where the sample size $n$ and dimension $p$ are comparable, with $n, p \rightarrow \infty $ and $\gamma_n = p/n \rightarrow \gamma > 0$. Yet, many datasets -- perhaps most -- have very different numbers of rows and columns. We consider instead the disproportional-growth asymptotic framework, where $n, p \rightarrow \infty$ and $\gamma_n \rightarrow 0$ or $\gamma_n \rightarrow \infty$. Either disproportional limit induces novel behavior unseen within previous proportional and fixed-$p$ analyses. We study the spiked covariance model, with theoretical covariance a low-rank perturbation of the identity. For each of 15 different loss functions, we exhibit in closed form new optimal shrinkage and thresholding rules. Our optimal procedures demand extensive eigenvalue shrinkage and offer substantial performance benefits over the standard empirical covariance estimator. Practitioners may ask whether to view their data as arising within (and apply the procedures of) the proportional or disproportional frameworks. Conveniently, it is possible to remain {\it framework agnostic}: one unified set of closed-form shrinkage rules (depending only on the aspect ratio $\gamma_n$ of the given data) offers full asymptotic optimality under either framework. At the heart of the phenomena we explore is the spiked Wigner model, in which a low-rank matrix is perturbed by symmetric noise. Exploiting a connection to the spiked covariance model as $\gamma_n \rightarrow 0$, we derive optimal eigenvalue shrinkage rules for estimation of the low-rank component, of independent and fundamental interest.

Nonlinear Shrinkage of the Covariance Matrix for Portfolio Selection: Markowitz Meets Goldilocks

Covariance Matrix Analysis for Optimal Portfolio Selection

Asymptotic Behavior of Mean-Cvar Portfolio Selection Model under Nonparametric Framework

Optimal shrinkage-based portfolio selection in high dimensions

Multi Anchor Point Shrinkage for the Sample Covariance Matrix (Extended Version)

Covariance matrix filtering and portfolio optimisation: the Average Oracle vs Non-Linear Shrinkage and all the variants of DCC-NLS

Nonlinear shrinkage test on a large‐dimensional covariance matrix

Shrinkage MMSE estimators of covariances beyond the zero-mean and stationary variance assumptions

Shrinkage regression for multivariate inference with missing data, and an application to portfolio balancing

A Robust Statistics Approach to Minimum Variance Portfolio Optimization

A Non-Parametric Shrinkage Mean Estimation for Arbitrary Quadratic Loss Functions and Unknown Covariance Matrices

Cross-validation based Nonlinear Shrinkage

Estimation of Large Financial Covariances: A Cross-Validation Approach

Nonparametric Stein-type shrinkage covariance matrix estimators in high-dimensional settings

Non-parametric Shrinkage Mean Estimation for Quadratic Loss Functions with Unknown Covariance Matrices

The Efficient Frontier for Weakly Correlated Assets

Optimal Eigenvalue Shrinkage in the Semicircle Limit

Portfolio Optimization with Robust Covariance and Conditional Value-at-Risk Constraints

Mean-variance portfolio selection with nonlinear wealth dynamics and random coefficients

Target selection in shrinkage estimation of covariance matrix: A structural similarity approach