Abstract:We suggest a method for estimating a covariance matrix on the basis of a sample of vectors drawn from a multivariate normal distribution. In particular, we penalize the likelihood with a lasso penalty on the entries of the covariance matrix. This penalty plays two important roles: it reduces the effective number of parameters, which is important even when the dimension of the vectors is smaller than the sample size since the number of parameters grows quadratically in the number of variables, and it produces an estimate which is sparse. In contrast to sparse inverse covariance estimation, our method's close relative, the sparsity attained here is in the covariance matrix itself rather than in the inverse matrix. Zeros in the covariance matrix correspond to marginal independencies; thus, our method performs model selection while providing a positive definite estimate of the covariance. The proposed penalized maximum likelihood problem is not convex, so we use a majorize-minimize approach in which we iteratively solve convex approximations to the original nonconvex problem. We discuss tuning parameter selection and demonstrate on a flow-cytometry dataset how our method produces an interpretable graphical display of the relationship between variables. We perform simulations that suggest that simple elementwise thresholding of the empirical covariance matrix is competitive with our method for identifying the sparsity structure. Additionally, we show how our method can be used to solve a previously studied special case in which a desired sparsity pattern is prespecified.

The statistical rate for support matrix machines under low rankness and row (column) sparsity

Robust Regularized Low-Rank Matrix Models for Regression and Classification

Regularized Matrix Regression

Sparse and Low-Rank Covariance Matrix Estimation

Low-rank matrix estimation via nonconvex spectral regularized methods in errors-in-variables matrix regression

A framework of regularized low-rank matrix models for regression and classification

Support matrix machine: exploring sample sparsity, low rank, and adaptive sieving in high-performance computing

Sparse Support Matrix Machine

Unified Low-Rank Matrix Estimate Via Penalized Matrix Least Squares Approximation

Estimation of Sparse Covariance Matrix Via Non-Convex Regularization

Sparse estimation of a covariance matrix

A Constrained L1 Minimization Approach to Sparse Precision Matrix Estimation

Low-rank matrix estimation in multi-response regression with measurement errors: Statistical and computational guarantees.

Estimation of high-dimensional low-rank matrices

Sparse Least Squares Low Rank Kernel Machines

Optimal Rates of Convergence for Sparse Covariance Matrix Estimation

Efficient Low Rank Matrix Approximation Via Orthogonality Pursuit and ℓ2 Regularization

Minimax Optimal Estimation of High-Dimensional Sparse Covariance Matrices with Missing Data.

Asymptotic Properties of Lasso+mLS and Lasso+Ridge in Sparse High-Dimensional Linear Regression

The linearized alternating direction method of multipliers for low-rank and fused LASSO matrix regression model

Fast Low-Rank Matrix Learning with Nonconvex Regularization