Abstract:We consider the problem of learning a sparse graph under the Laplacian constrained Gaussian graphical models. This problem can be formulated as a penalized maximum likelihood estimation of the Laplacian constrained precision matrix. Like in the classical graphical lasso problem, recent works made use of the $\ell_1$-norm regularization with the goal of promoting sparsity in Laplacian constrained precision matrix estimation. However, we find that the widely used $\ell_1$-norm is not effective in imposing a sparse solution in this problem. Through empirical evidence, we observe that the number of nonzero graph weights grows with the increase of the regularization parameter. From a theoretical perspective, we prove that a large regularization parameter will surprisingly lead to a complete graph, i.e., every pair of vertices is connected by an edge. To address this issue, we introduce the nonconvex sparsity penalty, and propose a new estimator by solving a sequence of weighted $\ell_1$-norm penalized sub-problems. We establish the non-asymptotic optimization performance guarantees on both optimization error and statistical error, and prove that the proposed estimator can recover the edges correctly with a high probability. To solve each sub-problem, we develop a projected gradient descent algorithm which enjoys a linear convergence rate. Finally, an extension to learn disconnected graphs is proposed by imposing additional rank constraint. We propose a numerical algorithm based on based on the alternating direction method of multipliers, and establish its theoretical sequence convergence. Numerical experiments involving synthetic and real-world data sets demonstrate the effectiveness of the proposed method.

Sparse Norm-Regularized Reconstructive Coefficients Learning

Sparse representation via L2,p norm for image classification

Simultaneously Sparse and Low-Rank Matrix Reconstruction Via Nonconvex and Nonseparable Regularization

An Adaptive Regularization Method for Sparse Representation

l1-norm regularized l1-norm best-fit lines

A Recursive Least Squares Algorithm with ℓ 1 Regularization for Sparse Representation

Class-specific Adaptive Sparse Representation Via Reweighted ℓ1 Norm Minimization for Image Classification.

Learning Sparse Visual Representations with Leaky Capped Norm Regularizers

Collaborative Representation for Classification, Sparse or Non-sparse?

A Re-Weighted Smoothed L0 -Norm Regularized Sparse Reconstructed Algorithm for Linear Inverse Problems

L1-norm Laplacian Support Vector Machine for Data Reduction in Semi-Supervised Learning

Overcomplete Dictionary Learning For Nonnegative Sparse Representation With An (Lp)-Norm Constraint Based On Majorize-Minimization

Iteratively Re-Weighted Method for Sparsity-Inducing Norms

Sparse Reconstruction by Means of the Standard Tikhonov Regularization

Does the $\ell_1$-norm Learn a Sparse Graph under Laplacian Constrained Graphical Models?

A New Smoothed L0 Regularization Approach for Sparse Signal Recovery

Sparse Signal Recovery Via Minimax-Concave Penalty and ℓ 1 -Norm Loss Function.

Linear Programming-Based Sparse Kernel Regression with L1-Norm Minimization for Nonlinear System Modeling

Efficient and Robust Recovery of Sparse Signal and Image Using Generalized Nonconvex Regularization

Constrained Analysis Dictionary Learning with the L(1/2)-Norm Regularizer

Learning Deep Sparse Regularizers with Applications to Multi-View Clustering and Semi-Supervised Classification