Abstract:We consider the covariance selection problem where variables are clustered into groups and the inverse covariance matrix is expected to have a blockwise sparse structure. This problem is realized via penalizing the maximum likelihood estimation of the inverse covariance matrix by group Lasso regularization. We propose to solve the resulting log-determinant optimization problem with the classical proximal point algorithm (PPA). At each iteration, as it is difficult to update the primal variables directly, we first solve the dual subproblem by an inexact semismooth Newton-CG method and then update the primal variables by explicit formulas based on the computed dual variables. We also propose to accelerate the PPA by an inexact generalized Newton's method when the iterate is close to the solution. Theoretically, we prove that at the optimal solution, the nonsingularity of the generalized Hessian matrices of the dual subproblem is equivalent to the constraint nondegeneracy condition for the primal problem. Global and local convergence results are also presented for the proposed PPA. Moreover, based on the augmented Lagrangian function of the dual problem we derive an alternating direction method (ADM), which is easily implementable and is demonstrated to be efficient for random problems. Numerical results, including comparisons with the ADM on both synthetic and real data, are presented to demonstrate that the proposed Newton-CG based PPA is stable and efficient and, in particular, outperforms the ADM when high accuracy is required.

Partitioning group correction Cholesky techniques for large scale sparse unconstrained optimization

On the convergence of partitioning group correction algorithms

The Substitution Secant/Finite Difference Method for Large Scale Sparse Unconstrained Optimization

An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems

A Partially Parallel Prediction-Correction Splitting Method for Convex Optimization Problems with Separable Structure

A Parallel Line Search Subspace Correction Method for Composite Convex Optimization

Preconditioned Bi-conjugate gradient method of large-scale sparse complex linear equation group

Computing optimal partition problems via Lagrange multiplier approach

On generalized local Hermitian and skew-Hermitian splitting iterative method for block two-by-two linear systems.

Exact Algorithm for Concave Knapsack Problems: Linear Underestimation and Partition Method

A Distributed Algorithm for High-Dimension Convex Quadratically Constrained Quadratic Programs

An Improved ICCG Method of Large-Scale Sparse Linear Equation Group

A Proximal Point Algorithm For Log-Determinant Optimization With Group Lasso Regularization

RESTRICTIVELY PRECONDITIONED CONJUGATE GRADIENT METHOD FOR A SERIES OF CONSTANTLY AUGMENTED LEAST SQUARES PROBLEMS

Efficient sparse Hessian based algorithms for the clustered lasso problem.

A Partially Parallel Splitting Method for Multiple-Block Separable Convex Programming with Applications to Robust Pca

Strong Partitioning and a Machine Learning Approximation for Accelerating the Global Optimization of Nonconvex QCQPs

A block Lanczos method for large-scale quadratic minimization problems with orthogonality constraints

Subspace Newton method for sparse group optimization problem

Predictor-Corrector Smoothing Methods for Monotone LCP

Complex incomplete cholesky factorization preconditioned Bi-conjugate gradient method