Abstract:High dimensional covariance estimation and graphical models is a contemporary topic in statistics and machine learning having widespread applications. An important line of research in this regard is to shrink the extreme spectrum of the covariance matrix estimators. A separate line of research in the literature has considered sparse inverse covariance estimation which in turn gives rise to graphical models. In practice, however, a sparse covariance or inverse covariance matrix which is simultaneously well-conditioned and at the same time computationally tractable is desired. There has been little research at the confluence of these three topics. In this paper we consider imposing a condition number constraint to various types of losses used in covariance and inverse covariance matrix estimation. When the loss function can be decomposed as a sum of an orthogonally invariant function of the estimate and its inner product with a function of the sample covariance matrix, we show that a solution path algorithm can be derived, involving a series of ordinary differential equations. The path algorithm is attractive because it provides the entire family of estimates for all possible values of the condition number bound, at the same computational cost of a single estimate with a fixed upper bound. An important finding is that the proximal operator for the condition number constraint, which turns out to be very useful in regularizing loss functions that are not orthogonally invariant and may yield non-positive-definite estimates, can be efficiently computed by this path algorithm. As a concrete illustration of its practical importance, we develop an operator-splitting algorithm that imposes a guarantee of well-conditioning as well as positive definiteness to recently proposed convex pseudo-likelihood based graphical model selection methods.

Spurious negative eigenvalues of numerical variance-covariance matrices in many-body systems correlate with the existence of frozen degrees of freedom

EIGENVALUE DISTRIBUTIONS OF VARIANCE COMPONENTS ESTIMATORS IN HIGH-DIMENSIONAL RANDOM EFFECTS MODELS

Conservation laws and spin system modeling through principal component analysis

From principal component to direct coupling analysis of coevolution in proteins: Low-eigenvalue modes are needed for structure prediction

On Eigenvalues of a High-Dimensional Spatial-Sign Covariance Matrix

Fortuitous Correlations in Molecular Dynamics Simulations: Their Harmful Influence on the Probability Distributions of the Main Principal Components

A Result About the Classification of Quantum Covariance Matrices Based on Their Eigenspectra

Statistical physics of principal minors: Cavity approach

The Quadratic Optimization Bias Of Large Covariance Matrices

A D.C. Programming Approach to the Sparse Generalized Eigenvalue Problem

Inflationary dynamics for matrix eigenvalue problems

Theory of functional principal component analysis for discretely observed data

Matrix rigidity and the ill-posedness of Robust PCA and matrix completion

Dynamic Principal Subspaces with Sparsity in High Dimensions

Dynamic Principal Subspaces in High Dimensions

High-Dimensional Block Diagonal Covariance Structure Detection Using Singular Vectors

When and why are principal component scores a good tool for visualizing high-dimensional data?

Orthogonal Sparse PCA and Covariance Estimation via Procrustes Reformulation

Towards a sparse, scalable, and stably positive definite (inverse) covariance estimator

Time-Averaged Normal Coordinate Analysis Of Polymer Particles And Crystals