Abstract:Modern datasets are trending towards ever higher dimension. In response, recent theoretical studies of covariance estimation often assume the proportional-growth asymptotic framework, where the sample size $n$ and dimension $p$ are comparable, with $n, p \rightarrow \infty $ and $\gamma_n = p/n \rightarrow \gamma > 0$. Yet, many datasets -- perhaps most -- have very different numbers of rows and columns. We consider instead the disproportional-growth asymptotic framework, where $n, p \rightarrow \infty$ and $\gamma_n \rightarrow 0$ or $\gamma_n \rightarrow \infty$. Either disproportional limit induces novel behavior unseen within previous proportional and fixed-$p$ analyses. We study the spiked covariance model, with theoretical covariance a low-rank perturbation of the identity. For each of 15 different loss functions, we exhibit in closed form new optimal shrinkage and thresholding rules. Our optimal procedures demand extensive eigenvalue shrinkage and offer substantial performance benefits over the standard empirical covariance estimator. Practitioners may ask whether to view their data as arising within (and apply the procedures of) the proportional or disproportional frameworks. Conveniently, it is possible to remain {\it framework agnostic}: one unified set of closed-form shrinkage rules (depending only on the aspect ratio $\gamma_n$ of the given data) offers full asymptotic optimality under either framework. At the heart of the phenomena we explore is the spiked Wigner model, in which a low-rank matrix is perturbed by symmetric noise. Exploiting a connection to the spiked covariance model as $\gamma_n \rightarrow 0$, we derive optimal eigenvalue shrinkage rules for estimation of the low-rank component, of independent and fundamental interest.

Frequentist Shrinkage under Inequality Constraints

Bias Reduction in Instrumental Variable Estimation through First-Stage Shrinkage

Frequentist Model Averaging under Inequality Constraints

Shrinkage MMSE estimators of covariances beyond the zero-mean and stationary variance assumptions

A constrained risk inequality for general losses

Shrinkage for Categorical Regressors

Penalized bias reduction in extreme value estimation for censored Pareto-type data, and long-tailed insurance applications

A Non-Parametric Shrinkage Mean Estimation for Arbitrary Quadratic Loss Functions and Unknown Covariance Matrices

Shrinkage estimation of rate statistics

Bayes estimator for the inequality-constrained regression model*

A Geometric Unification of Distributionally Robust Covariance Estimators: Shrinking the Spectrum by Inflating the Ambiguity Set

Shrinkage estimators in zero-inflated Bell regression model with application

Shrinkage for Extreme Partial Least-Squares

Shrinkage Methods for Treatment Choice

Optimal Eigenvalue Shrinkage in the Semicircle Limit

Identification-robust methods for comparing inequality with an application to regional disparities

Maximum likelihood estimation for Gaussian processes under inequality constraints

Bayesian Estimation Approach for Linear Regression Models with Linear Inequality Restrictions

Linear Regression Analysis with Inequality Constraints on the Regression Parameters Via Empirical Likelihood

Shrinkage estimation in the zero-inflated Poisson regression model with right-censored data

Nonparametric Stein-type shrinkage covariance matrix estimators in high-dimensional settings