Abstract:We suggest a method for estimating a covariance matrix on the basis of a sample of vectors drawn from a multivariate normal distribution. In particular, we penalize the likelihood with a lasso penalty on the entries of the covariance matrix. This penalty plays two important roles: it reduces the effective number of parameters, which is important even when the dimension of the vectors is smaller than the sample size since the number of parameters grows quadratically in the number of variables, and it produces an estimate which is sparse. In contrast to sparse inverse covariance estimation, our method's close relative, the sparsity attained here is in the covariance matrix itself rather than in the inverse matrix. Zeros in the covariance matrix correspond to marginal independencies; thus, our method performs model selection while providing a positive definite estimate of the covariance. The proposed penalized maximum likelihood problem is not convex, so we use a majorize-minimize approach in which we iteratively solve convex approximations to the original nonconvex problem. We discuss tuning parameter selection and demonstrate on a flow-cytometry dataset how our method produces an interpretable graphical display of the relationship between variables. We perform simulations that suggest that simple elementwise thresholding of the empirical covariance matrix is competitive with our method for identifying the sparsity structure. Additionally, we show how our method can be used to solve a previously studied special case in which a desired sparsity pattern is prespecified.

Covariance Operator Estimation via Adaptive Thresholding

Adaptive Thresholding for Sparse Covariance Matrix Estimation

Adaptive Functional Thresholding for Sparse Covariance Function Estimation in High Dimensions

Optimal Estimation of Structured Covariance Operators

Covariance regularization by thresholding

Large Covariance Estimation by Thresholding Principal Orthogonal Complements

Minimax Optimal Estimation of High-Dimensional Sparse Covariance Matrices with Missing Data.

Adaptive Covariance Estimation with model selection

Separable Expansions for Covariance Estimation

A Class of Optimal Estimators for the Covariance Operator in Reproducing Kernel Hilbert Spaces

Large-Dimensional Positive Definite Covariance Estimation for High Frequency Data via Low-rank and Sparse Matrix Decomposition

Random Surface Covariance Estimation by Shifted Partial Tracing

One-Bit Covariance Reconstruction with Non-zero Thresholds: Algorithm and Performance Analysis

A variational Bayes framework for sparse adaptive estimation

SHARP OPTIMALITY FOR HIGH-DIMENSIONAL COVARIANCE TESTING UNDER SPARSE SIGNALS

Covariance estimation for multivariate conditionally Gaussian dynamic linear models

Adaptive robust estimation in sparse vector model

Detecting Spectral Breaks in Spiked Covariance Models

Covariance Matrix Estimation from Linearly-Correlated Gaussian Samples

Sparse estimation of a covariance matrix

Estimation of Conditional Mean Operator under the Bandable Covariance Structure