Abstract:High dimensional covariance estimation and graphical models is a contemporary topic in statistics and machine learning having widespread applications. An important line of research in this regard is to shrink the extreme spectrum of the covariance matrix estimators. A separate line of research in the literature has considered sparse inverse covariance estimation which in turn gives rise to graphical models. In practice, however, a sparse covariance or inverse covariance matrix which is simultaneously well-conditioned and at the same time computationally tractable is desired. There has been little research at the confluence of these three topics. In this paper we consider imposing a condition number constraint to various types of losses used in covariance and inverse covariance matrix estimation. When the loss function can be decomposed as a sum of an orthogonally invariant function of the estimate and its inner product with a function of the sample covariance matrix, we show that a solution path algorithm can be derived, involving a series of ordinary differential equations. The path algorithm is attractive because it provides the entire family of estimates for all possible values of the condition number bound, at the same computational cost of a single estimate with a fixed upper bound. An important finding is that the proximal operator for the condition number constraint, which turns out to be very useful in regularizing loss functions that are not orthogonally invariant and may yield non-positive-definite estimates, can be efficiently computed by this path algorithm. As a concrete illustration of its practical importance, we develop an operator-splitting algorithm that imposes a guarantee of well-conditioning as well as positive definiteness to recently proposed convex pseudo-likelihood based graphical model selection methods.

Nonparametric matrix regression function estimation over symmetric positive definite matrices

Kernel Methods on the Riemannian Manifold of Symmetric Positive Definite Matrices

A Spatial Bayesian Semiparametric Mixture Model for Positive Definite Matrices with Applications to Diffusion Tensor Imaging

Additive Models for Symmetric Positive-Definite Matrices, Riemannian Manifolds and Lie groups

Sparse Coding and Dictionary Learning for Symmetric Positive Definite Matrices: A Kernel Approach

Manifold Kernel Sparse Representation of Symmetric Positive-Definite Matrices and Its Applications

A Non-Parametric Shrinkage Mean Estimation for Arbitrary Quadratic Loss Functions and Unknown Covariance Matrices

Large-Dimensional Positive Definite Covariance Estimation for High Frequency Data via Low-rank and Sparse Matrix Decomposition

Geostatistical Modeling of Positive Definite Matrices: An Application to Diffusion Tensor Imaging

A calibration method for non-positive definite covariance matrix in multivariate data analysis

Modeling Symmetric Positive Definite Matrices with An Application to Functional Brain Connectivity

An Empirical Bayes Approach to Shrinkage Estimation on the Manifold of Symmetric Positive-Definite Matrices

Principal regression for high dimensional covariance matrices

Robust Recovery of Myocardial Kinematics Using Dual ℋ∞$\mathcal {h}_{\infty }$ Criteria

Statistical inference for wavelet curve estimators of symmetric positive definite matrices

Multivariate Intrinsic Local Polynomial Regression on Isometric Riemannian Manifolds: Applications to Positive Definite Data

Multifidelity Covariance Estimation via Regression on the Manifold of Symmetric Positive Definite Matrices

Efficient Algorithms for Positive Semi-Definite Total Least Squares Problems, Minimum Rank Problem and Correlation Matrix Computation

Towards a sparse, scalable, and stably positive definite (inverse) covariance estimator

Barycenter Estimation of Positive Semi-Definite Matrices with Bures-Wasserstein Distance

Intrinsic Wavelet Regression for Curves of Hermitian Positive Definite Matrices