Abstract:Neural networks are central to many emerging technologies, but verifying their correctness remains a major challenge. It is known that network outputs can be sensitive and fragile to even small input perturbations, thereby increasing the risk of unpredictable and undesirable behavior. Fast and accurate verification of neural networks is therefore critical to their widespread adoption, and in recent years, various methods have been developed as a response to this problem. In this paper, we focus on improving semidefinite programming (SDP) based techniques for neural network verification. Such techniques offer the power of expressing complex geometric constraints while retaining a convex problem formulation, but scalability remains a major issue in practice. Our starting point is the DeepSDP framework proposed by Fazlyab et al., which uses quadratic constraints to abstract the verification problem into a large-scale SDP. However, solving this SDP quickly becomes intractable when the network grows. Our key observation is that by leveraging chordal sparsity, we can decompose the primary computational bottleneck of DeepSDP -- a large linear matrix inequality (LMI) -- into an equivalent collection of smaller LMIs. We call our chordally sparse optimization program Chordal-DeepSDP and prove that its construction is identically expressive as that of DeepSDP. Moreover, we show that additional analysis of Chordal-DeepSDP allows us to further rewrite its collection of LMIs in a second level of decomposition that we call Chordal-DeepSDP-2 -- which results in another significant computational gain. Finally, we provide numerical experiments on real networks of learned cart-pole dynamics, showcasing the computational advantage of Chordal-DeepSDP and Chordal-DeepSDP-2 over DeepSDP.

Efficient Neural Network Verification via Layer-based Semidefinite Relaxations and Linear Cuts

Tight Neural Network Verification via Semidefinite Relaxations and Linear Reformulations

Improving Neural Network Verification through Spurious Region Guided Refinement.

A Dual Relaxation Method for Neural Network Verification

Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming

Efficient Complete Verification of Neural Networks Via Layerwised Splitting and Refinement

Towards Optimal Branching of Linear and Semidefinite Relaxations for Neural Network Robustness Certification

Chordal Sparsity for SDP-based Neural Network Verification

Bound Tightening using Rolling-Horizon Decomposition for Neural Network Verification

A Dual Approach to Scalable Verification of Deep Networks

kProp: Multi-neuron Relaxation Method for Neural Network Robustness Verification.

Tight Certification of Adversarially Trained Neural Networks via Nonconvex Low-Rank Semidefinite Relaxations

Achieving the Tightest Relaxation of Sigmoids for Formal Verification

Semidefinite relaxations for certifying robustness to adversarial examples

Automated Design of Linear Bounding Functions for Sigmoidal Nonlinearities in Neural Networks

Efficient verification of neural networks based on neuron branching and LP abstraction

Verification of Geometric Robustness of Neural Networks via Piecewise Linear Approximation and Lipschitz Optimisation

Scaling the Convex Barrier with Sparse Dual Algorithms

An RNN-Based Framework for the MILP Problem in Robustness Verification of Neural Networks.

Expediting Neural Network Verification via Network Reduction

Tightening Robustness Verification of Convolutional Neural Networks with Fine-Grained Linear Approximation.