Abstract:High dimensional covariance estimation and graphical models is a contemporary topic in statistics and machine learning having widespread applications. An important line of research in this regard is to shrink the extreme spectrum of the covariance matrix estimators. A separate line of research in the literature has considered sparse inverse covariance estimation which in turn gives rise to graphical models. In practice, however, a sparse covariance or inverse covariance matrix which is simultaneously well-conditioned and at the same time computationally tractable is desired. There has been little research at the confluence of these three topics. In this paper we consider imposing a condition number constraint to various types of losses used in covariance and inverse covariance matrix estimation. When the loss function can be decomposed as a sum of an orthogonally invariant function of the estimate and its inner product with a function of the sample covariance matrix, we show that a solution path algorithm can be derived, involving a series of ordinary differential equations. The path algorithm is attractive because it provides the entire family of estimates for all possible values of the condition number bound, at the same computational cost of a single estimate with a fixed upper bound. An important finding is that the proximal operator for the condition number constraint, which turns out to be very useful in regularizing loss functions that are not orthogonally invariant and may yield non-positive-definite estimates, can be efficiently computed by this path algorithm. As a concrete illustration of its practical importance, we develop an operator-splitting algorithm that imposes a guarantee of well-conditioning as well as positive definiteness to recently proposed convex pseudo-likelihood based graphical model selection methods.

Non-parametric Shrinkage Mean Estimation for Quadratic Loss Functions with Unknown Covariance Matrices

A Non-Parametric Shrinkage Mean Estimation for Arbitrary Quadratic Loss Functions and Unknown Covariance Matrices

A Shrinkage Approach to Joint Estimation of Multiple Covariance Matrices

Ledoit-Wolf linear shrinkage with unknown mean

Estimation of Sparse Covariance Matrix Via Non-Convex Regularization

Regularized Estimation of Covariance Structure Through Quadratic Loss Function

Shrinkage MMSE estimators of covariances beyond the zero-mean and stationary variance assumptions

Shrinkage Estimation for the Diagonal Multivariate Exponential Families

A Class of Optimal Estimators for the Covariance Operator in Reproducing Kernel Hilbert Spaces

Towards a sparse, scalable, and stably positive definite (inverse) covariance estimator

Improved estimation of the MSEs and the MSE matrices for shrinkage estimators of multivariate normal means and their applications

Minimax Optimal Estimation of High-Dimensional Sparse Covariance Matrices with Missing Data.

Asymptotic Covariance Estimation by Gaussian Random Perturbation

Nonparametric Stein-type shrinkage covariance matrix estimators in high-dimensional settings

Shrinkage for Categorical Regressors

Nonnegative Estimation and Variable Selection under Minimax Concave Penalty for Sparse High-Dimensional Linear Regression Models

Linear shrinkage estimation of covariance matrices using low-complexity cross-validation

Matrix quadratic risk of orthogonally invariant estimators for a normal mean matrix

Sparse and Low-Rank Matrix Quantile Estimation with Application to Quadratic Regression

Sparse and Low-Rank Covariance Matrix Estimation

Positive Definite Estimation of Large Covariance Matrix Using Generalized Nonconvex Penalties.