Abstract:Neural networks are central to many emerging technologies, but verifying their correctness remains a major challenge. It is known that network outputs can be sensitive and fragile to even small input perturbations, thereby increasing the risk of unpredictable and undesirable behavior. Fast and accurate verification of neural networks is therefore critical to their widespread adoption, and in recent years, various methods have been developed as a response to this problem. In this paper, we focus on improving semidefinite programming (SDP) based techniques for neural network verification. Such techniques offer the power of expressing complex geometric constraints while retaining a convex problem formulation, but scalability remains a major issue in practice. Our starting point is the DeepSDP framework proposed by Fazlyab et al., which uses quadratic constraints to abstract the verification problem into a large-scale SDP. However, solving this SDP quickly becomes intractable when the network grows. Our key observation is that by leveraging chordal sparsity, we can decompose the primary computational bottleneck of DeepSDP -- a large linear matrix inequality (LMI) -- into an equivalent collection of smaller LMIs. We call our chordally sparse optimization program Chordal-DeepSDP and prove that its construction is identically expressive as that of DeepSDP. Moreover, we show that additional analysis of Chordal-DeepSDP allows us to further rewrite its collection of LMIs in a second level of decomposition that we call Chordal-DeepSDP-2 -- which results in another significant computational gain. Finally, we provide numerical experiments on real networks of learned cart-pole dynamics, showcasing the computational advantage of Chordal-DeepSDP and Chordal-DeepSDP-2 over DeepSDP.

Complexity of Chordal Conversion for Sparse Semidefinite Programs with Small Treewidth

Exploring chordal sparsity in semidefinite programming with sparse plus low-rank data matrices

Exploiting Chordal Sparsity for Fast Global Optimality with Application to Localization

Chordal Decomposition in Rank Minimized Semidefinite Programs with Applications to Subspace Clustering

Chordal Sparsity for SDP-based Neural Network Verification

Decomposition Methods for Large-Scale Semidefinite Programs with Chordal Aggregate Sparsity and Partial Orthogonality

Degeneracy in Maximal Clique Decomposition for Semidefinite Programs

Fast ADMM for Semidefinite Programs with Chordal Sparsity

Chordal Decomposition in Operator-Splitting Methods for Sparse Semidefinite Programs

Chordal and Factor-Width Decompositions for Scalable Semidefinite and Polynomial Optimization.

A New Global Algorithm for Max-Cut Problem with Chordal Sparsity

Decomposed Structured Subsets for Semidefinite Optimization

A Two-Step Pre-Processing for Semidefinite Programming

An Enhanced Semidefinite Relaxation Model Combined with Clique Graph Merging Strategy for Efficient AC Optimal Power Flow Solution

Chordal Sparsity for Lipschitz Constant Estimation of Deep Neural Networks

Sum-of-squares Chordal Decomposition of Polynomial Matrix Inequalities

Scalable Design of Structured Controllers Using Chordal Decomposition

Suboptimality bounds for trace-bounded SDPs enable a faster and scalable low-rank SDP solver SDPLR+

Polynomial Integrality Gaps for Strong SDP Relaxations of Densest K-Subgraph

On the Equivalence of SDP Feasibility and a Convex Hull Relaxation for System of Quadratic Equations

Structured Semidefinite Programming for Recovering Structured Preconditioners