Abstract:We consider the problem of learning a sparse graph under the Laplacian constrained Gaussian graphical models. This problem can be formulated as a penalized maximum likelihood estimation of the Laplacian constrained precision matrix. Like in the classical graphical lasso problem, recent works made use of the $\ell_1$-norm regularization with the goal of promoting sparsity in Laplacian constrained precision matrix estimation. However, we find that the widely used $\ell_1$-norm is not effective in imposing a sparse solution in this problem. Through empirical evidence, we observe that the number of nonzero graph weights grows with the increase of the regularization parameter. From a theoretical perspective, we prove that a large regularization parameter will surprisingly lead to a complete graph, i.e., every pair of vertices is connected by an edge. To address this issue, we introduce the nonconvex sparsity penalty, and propose a new estimator by solving a sequence of weighted $\ell_1$-norm penalized sub-problems. We establish the non-asymptotic optimization performance guarantees on both optimization error and statistical error, and prove that the proposed estimator can recover the edges correctly with a high probability. To solve each sub-problem, we develop a projected gradient descent algorithm which enjoys a linear convergence rate. Finally, an extension to learn disconnected graphs is proposed by imposing additional rank constraint. We propose a numerical algorithm based on based on the alternating direction method of multipliers, and establish its theoretical sequence convergence. Numerical experiments involving synthetic and real-world data sets demonstrate the effectiveness of the proposed method.

Structure Learning of Sparse Directed Acyclic Graphs Incorporating the Scale-Free Property

Learning directed acyclic graphs based on sparsest permutations

Does the $\ell_1$-norm Learn a Sparse Graph under Laplacian Constrained Graphical Models?

Sparse Directed Acyclic Graphs Incorporating the Covariates

Network Topology Inference with Sparsity and Laplacian Constraints

Analyzing and Learning Sparse and Scale-Free Networks Using Gaussian Graphical Models.

Dense Graphs with Scale-Free Feature.

Learning Undirected Graphical Models with Structure Penalty

Structured Sparsity With Group-Graph Regularization

Efficient Learning of Nonparametric Directed Acyclic Graph with Statistical Guarantee

Dense Networks with Scale-Free Feature

On Low Rank Directed Acyclic Graphs and Causal Structure Learning

A first-order optimization algorithm for statistical learning with hierarchical sparsity structure

Representation Learning for Scale-free Networks

Learning Scale-Free Networks by Dynamic Node-Specific Degree Prior

Scalable Structure Learning for Sparse Context-Specific Systems

Group And Graph Joint Sparsity For Linked Data Classification

Coordinated Multi-Neighborhood Learning on a Directed Acyclic Graph

L-1-Graph Construction Using Structured Sparsity

A Unified Approach to Scalable Spectral Sparsification of Directed Graphs

Fast and scalable learning of sparse changes in high-dimensional graphical model structure