Abstract:We consider the covariance selection problem where variables are clustered into groups and the inverse covariance matrix is expected to have a blockwise sparse structure. This problem is realized via penalizing the maximum likelihood estimation of the inverse covariance matrix by group Lasso regularization. We propose to solve the resulting log-determinant optimization problem with the classical proximal point algorithm (PPA). At each iteration, as it is difficult to update the primal variables directly, we first solve the dual subproblem by an inexact semismooth Newton-CG method and then update the primal variables by explicit formulas based on the computed dual variables. We also propose to accelerate the PPA by an inexact generalized Newton's method when the iterate is close to the solution. Theoretically, we prove that at the optimal solution, the nonsingularity of the generalized Hessian matrices of the dual subproblem is equivalent to the constraint nondegeneracy condition for the primal problem. Global and local convergence results are also presented for the proposed PPA. Moreover, based on the augmented Lagrangian function of the dual problem we derive an alternating direction method (ADM), which is easily implementable and is demonstrated to be efficient for random problems. Numerical results, including comparisons with the ADM on both synthetic and real data, are presented to demonstrate that the proposed Newton-CG based PPA is stable and efficient and, in particular, outperforms the ADM when high accuracy is required.

A Fast Dual Projected Newton Method for L1-Regularized Least Squares

A fast dual projected newton method for ℓ1-regularized least squares

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

Dual Newton Proximal Point Algorithm for Solution Paths of the L1-Regularized Logistic Regression

A new penalized least absolute deviation model for high dimensional sparse linear regression and an efficient sequential linear programming algorithm

Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation.

Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation

Semismooth Newton Algorithm for Efficient Projections Onto ℓ1, ∞-Norm Ball

A Fast Coordinate Descent Method for High-Dimensional Non-Negative Least Squares using a Unified Sparse Regression Framework

A Dual Semismooth Newton Based Augmented Lagrangian Method for Large-Scale Linearly Constrained Sparse Group Square-Root Lasso Problems

A Unified Study on $L_1$ over $L_2$ Minimization

Parameterized proximal-gradient algorithms for L1/L2 sparse signal recovery

A Finite Newton Algorithm for Non-degenerate Piecewise Linear Systems.

A Proximal Point Algorithm For Log-Determinant Optimization With Group Lasso Regularization

l1-norm regularized l1-norm best-fit lines

Projected Newton method for large-scale Bayesian linear inverse problems

Semismooth Newton Algorithm for Efficient Projections onto $\ell_1, ∞$-norm Ball

A least distance estimator for a multivariate regression model using deep neural networks

Solving the L1 regularized least square problem via a box-constrained smooth minimization

A Fast and Scalable Pathwise-Solver for Group Lasso and Elastic Net Penalized Regression via Block-Coordinate Descent