Abstract:Neural networks are central to many emerging technologies, but verifying their correctness remains a major challenge. It is known that network outputs can be sensitive and fragile to even small input perturbations, thereby increasing the risk of unpredictable and undesirable behavior. Fast and accurate verification of neural networks is therefore critical to their widespread adoption, and in recent years, various methods have been developed as a response to this problem. In this paper, we focus on improving semidefinite programming (SDP) based techniques for neural network verification. Such techniques offer the power of expressing complex geometric constraints while retaining a convex problem formulation, but scalability remains a major issue in practice. Our starting point is the DeepSDP framework proposed by Fazlyab et al., which uses quadratic constraints to abstract the verification problem into a large-scale SDP. However, solving this SDP quickly becomes intractable when the network grows. Our key observation is that by leveraging chordal sparsity, we can decompose the primary computational bottleneck of DeepSDP -- a large linear matrix inequality (LMI) -- into an equivalent collection of smaller LMIs. We call our chordally sparse optimization program Chordal-DeepSDP and prove that its construction is identically expressive as that of DeepSDP. Moreover, we show that additional analysis of Chordal-DeepSDP allows us to further rewrite its collection of LMIs in a second level of decomposition that we call Chordal-DeepSDP-2 -- which results in another significant computational gain. Finally, we provide numerical experiments on real networks of learned cart-pole dynamics, showcasing the computational advantage of Chordal-DeepSDP and Chordal-DeepSDP-2 over DeepSDP.

Degeneracy in Maximal Clique Decomposition for Semidefinite Programs

Chordal Decomposition in Rank Minimized Semidefinite Programs with Applications to Subspace Clustering

Decomposition Methods for Large-Scale Semidefinite Programs with Chordal Aggregate Sparsity and Partial Orthogonality

Chordal Decomposition in Operator-Splitting Methods for Sparse Semidefinite Programs

Exploring chordal sparsity in semidefinite programming with sparse plus low-rank data matrices

Complexity of Chordal Conversion for Sparse Semidefinite Programs with Small Treewidth

Decomposed Structured Subsets for Semidefinite Optimization

Dealing with inequality constraints in large-scale semidefinite relaxations for graph coloring and maximum clique problems

Benign landscape for Burer-Monteiro factorizations of MaxCut-type semidefinite programs

Fast ADMM for Semidefinite Programs with Chordal Sparsity

A Two-Step Pre-Processing for Semidefinite Programming

Rank-sparsity decomposition for planted quasi clique recovery

Global Optimization for Cardinality-constrained Minimum Sum-of-Squares Clustering via Semidefinite Programming

A New Global Algorithm for Max-Cut Problem with Chordal Sparsity

Sum-of-squares Chordal Decomposition of Polynomial Matrix Inequalities

A feasible method for general convex low-rank SDP problems

Barrier and penalty methods for low-rank semidefinite programming with application to truss topology design

Chordal and Factor-Width Decompositions for Scalable Semidefinite and Polynomial Optimization.

Chordal Sparsity for SDP-based Neural Network Verification

Sparse Sub-gaussian Random Projections for Semidefinite Programming Relaxations

How Do Exponential Size Solutions Arise in Semidefinite Programming?