Abstract:We suggest a method for estimating a covariance matrix on the basis of a sample of vectors drawn from a multivariate normal distribution. In particular, we penalize the likelihood with a lasso penalty on the entries of the covariance matrix. This penalty plays two important roles: it reduces the effective number of parameters, which is important even when the dimension of the vectors is smaller than the sample size since the number of parameters grows quadratically in the number of variables, and it produces an estimate which is sparse. In contrast to sparse inverse covariance estimation, our method's close relative, the sparsity attained here is in the covariance matrix itself rather than in the inverse matrix. Zeros in the covariance matrix correspond to marginal independencies; thus, our method performs model selection while providing a positive definite estimate of the covariance. The proposed penalized maximum likelihood problem is not convex, so we use a majorize-minimize approach in which we iteratively solve convex approximations to the original nonconvex problem. We discuss tuning parameter selection and demonstrate on a flow-cytometry dataset how our method produces an interpretable graphical display of the relationship between variables. We perform simulations that suggest that simple elementwise thresholding of the empirical covariance matrix is competitive with our method for identifying the sparsity structure. Additionally, we show how our method can be used to solve a previously studied special case in which a desired sparsity pattern is prespecified.

Lower Bound Estimation for a Family of High-Dimensional Sparse Covariance Matrices

Minimax Optimal Estimation of High-Dimensional Sparse Covariance Matrices with Missing Data.

Optimal Rates of Convergence for Sparse Covariance Matrix Estimation

Sparse and Low-Rank Covariance Matrix Estimation

Estimation and confidence sets for sparse normal mixtures

Sparse estimation of a covariance matrix

Minimax Rates for High-dimensional Double Sparse Structure over $\ell_q$-balls

Estimation of Sparse Covariance Matrix Via Non-Convex Regularization

Distributional stability of sparse inverse covariance matrix estimators

Large-Dimensional Positive Definite Covariance Estimation for High Frequency Data via Low-rank and Sparse Matrix Decomposition

A Non-Parametric Shrinkage Mean Estimation for Arbitrary Quadratic Loss Functions and Unknown Covariance Matrices

Nonparametric estimation of large covariance matrices with conditional sparsity

Sparse Precision Matrix Estimation under Lower Polynomial Moment Assumption

High-Dimensional Covariance Estimation Via Constrained Lq-Type Regularization

High-probability minimax lower bounds

Sparse basis covariance matrix estimation for high dimensional compositional data via hard thresholding

Lower Bounds for Private Estimation of Gaussian Covariance Matrices under All Reasonable Parameter Regimes

A Constrained L1 Minimization Approach to Sparse Precision Matrix Estimation

CONFIDENCE INTERVALS FOR HIGH-DIMENSIONAL LINEAR REGRESSION: MINIMAX RATES AND ADAPTIVITY

A Shrinkage Approach to Joint Estimation of Multiple Covariance Matrices

On the upper and lower covariances under multiple probabilities