Abstract:High dimensional covariance estimation and graphical models is a contemporary topic in statistics and machine learning having widespread applications. An important line of research in this regard is to shrink the extreme spectrum of the covariance matrix estimators. A separate line of research in the literature has considered sparse inverse covariance estimation which in turn gives rise to graphical models. In practice, however, a sparse covariance or inverse covariance matrix which is simultaneously well-conditioned and at the same time computationally tractable is desired. There has been little research at the confluence of these three topics. In this paper we consider imposing a condition number constraint to various types of losses used in covariance and inverse covariance matrix estimation. When the loss function can be decomposed as a sum of an orthogonally invariant function of the estimate and its inner product with a function of the sample covariance matrix, we show that a solution path algorithm can be derived, involving a series of ordinary differential equations. The path algorithm is attractive because it provides the entire family of estimates for all possible values of the condition number bound, at the same computational cost of a single estimate with a fixed upper bound. An important finding is that the proximal operator for the condition number constraint, which turns out to be very useful in regularizing loss functions that are not orthogonally invariant and may yield non-positive-definite estimates, can be efficiently computed by this path algorithm. As a concrete illustration of its practical importance, we develop an operator-splitting algorithm that imposes a guarantee of well-conditioning as well as positive definiteness to recently proposed convex pseudo-likelihood based graphical model selection methods.

A Constrained L1 Minimization Approach to Sparse Precision Matrix Estimation

Sparse estimation of a covariance matrix

Optimal Rates of Convergence for Sparse Covariance Matrix Estimation

Sparse permutation invariant covariance estimation

Fast and Adaptive Sparse Precision Matrix Estimation in High Dimensions

Confidence Intervals for Sparse Precision Matrix Estimation Via Lasso Penalized D-trace Loss

Adjusting for High-Dimensional Covariates in Sparse Precision Matrix Estimation by Lasso Penalized D-Trace Loss

A fast ADMM algorithm for sparse precision matrix estimation using lasso penalized D-trace loss

A Unified Precision Matrix Estimation Framework Via Sparse Column-Wise Inverse Operator under Weak Sparsity

Positive-Definite Sparse Precision Matrix Estimation

Towards a sparse, scalable, and stably positive definite (inverse) covariance estimator

Nonparametric estimation of large covariance matrices with conditional sparsity

Adaptive Thresholding for Sparse Covariance Matrix Estimation

Orthogonal Sparse PCA and Covariance Estimation via Procrustes Reformulation

Concentration of a Sparse Bayesian Model With Horseshoe Prior in Estimating High‐Dimensional Precision Matrix

Estimation of Large Precision Matrices Through Block Penalization

Robust Sparse Recovery with Sparse Bernoulli matrices via Expanders

Joint Estimation Of Multiple High-Dimensional Precision Matrices

Covariate-adjusted Precision Matrix Estimation with an Application in Genetical Genomics

Fast and Positive Definite Estimation of Large Covariance Matrix for High-Dimensional Data Analysis

Concentration of a sparse Bayesian model with Horseshoe prior in estimating high-dimensional precision matrix