Abstract:High dimensional covariance estimation and graphical models is a contemporary topic in statistics and machine learning having widespread applications. An important line of research in this regard is to shrink the extreme spectrum of the covariance matrix estimators. A separate line of research in the literature has considered sparse inverse covariance estimation which in turn gives rise to graphical models. In practice, however, a sparse covariance or inverse covariance matrix which is simultaneously well-conditioned and at the same time computationally tractable is desired. There has been little research at the confluence of these three topics. In this paper we consider imposing a condition number constraint to various types of losses used in covariance and inverse covariance matrix estimation. When the loss function can be decomposed as a sum of an orthogonally invariant function of the estimate and its inner product with a function of the sample covariance matrix, we show that a solution path algorithm can be derived, involving a series of ordinary differential equations. The path algorithm is attractive because it provides the entire family of estimates for all possible values of the condition number bound, at the same computational cost of a single estimate with a fixed upper bound. An important finding is that the proximal operator for the condition number constraint, which turns out to be very useful in regularizing loss functions that are not orthogonally invariant and may yield non-positive-definite estimates, can be efficiently computed by this path algorithm. As a concrete illustration of its practical importance, we develop an operator-splitting algorithm that imposes a guarantee of well-conditioning as well as positive definiteness to recently proposed convex pseudo-likelihood based graphical model selection methods.

Variance Inflation Factor and Condition Number in multiple linear regression

Centered and non-centered variance inflation factor

The Quadratic Inference Functions in Measurement Error Model for Longitudinal Data

Extracting the Variance Inflation Factor and Other Multicollinearity Diagnostics from Typical Regression Results

A Redefined Variance Inflation Factor: Overcoming the Limitations of the Variance Inflation Factor

Evaluation of Variance Inflation Factors in Regression Models Using Latent Variable Modeling Methods

The VIF and MSE in Raise Regression

Condition numbers of the generalized ridge regression and its statistical estimation

Factor models and variable selection in high-dimensional regression analysis

Covariance estimation for multivariate conditionally Gaussian dynamic linear models

The Raise Regression: Justification, Properties and Application

Towards a sparse, scalable, and stably positive definite (inverse) covariance estimator

High-dimensional VARs with common factors

Estimation of ill-conditioned models using penalized sums of squares of the residuals

The Projected Covariance Measure for assumption-lean variable significance testing

A Study of Error Variance Estimation in Lasso Regression

Identifying the Number of Factors from Singular Values of A Large Sample Auto-Covariance Matrix

Diagnosing Multicollinearity of Logistic Regression Model

On mixed and componentwise condition numbers for Moore-Penrose inverse and linear least squares problems

Generalized varying coefficient partially linear measurement errors models

Inference for Conditional Value-at-risk of a Predictive Regression