Abstract:In this paper, we study certifying the robustness of ReLU neural networks against adversarial input perturbations. To diminish the relaxation error suffered by the popular linear programming (LP) and semidefinite programming (SDP) certification methods, we take a branch-and-bound approach to propose partitioning the input uncertainty set and solving the relaxations on each part separately. We show that this approach reduces relaxation error, and that the error is eliminated entirely upon performing an LP relaxation with a partition intelligently designed to exploit the nature of the ReLU activations. To scale this approach to large networks, we consider using a coarser partition whereby the number of parts in the partition is reduced. We prove that computing such a coarse partition that directly minimizes the LP relaxation error is NP-hard. By instead minimizing the worst-case LP relaxation error, we develop a closed-form branching scheme in the single-hidden layer case. We extend the analysis to the SDP, where the feasible set geometry is exploited to design a branching scheme that minimizes the worst-case SDP relaxation error. Experiments on MNIST, CIFAR-10, and Wisconsin breast cancer diagnosis classifiers demonstrate significant increases in the percentages of test samples certified. By independently increasing the input size and the number of layers, we empirically illustrate under which regimes the branched LP and branched SDP are best applied. Finally, we extend our LP branching method into a multi-layer branching heuristic, which attains comparable performance to prior state-of-the-art heuristics on large-scale, deep neural network certification benchmarks.

A Dual Relaxation Method for Neural Network Verification

Improving Neural Network Verification through Spurious Region Guided Refinement.

kProp: Multi-neuron Relaxation Method for Neural Network Robustness Verification.

Tight Neural Network Verification via Semidefinite Relaxations and Linear Reformulations

Efficient Neural Network Verification via Layer-based Semidefinite Relaxations and Linear Cuts

A Dual Approach to Scalable Verification of Deep Networks

Bound Tightening using Rolling-Horizon Decomposition for Neural Network Verification

Scaling the Convex Barrier with Sparse Dual Algorithms

Achieving the Tightest Relaxation of Sigmoids for Formal Verification

Harnessing Neuron Stability to Improve DNN Verification

Automated Design of Linear Bounding Functions for Sigmoidal Nonlinearities in Neural Networks

Multi-Neuron Unleashes Expressivity of ReLU Networks Under Convex Relaxation

Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming

Efficient verification of neural networks based on neuron branching and LP abstraction

An RNN-Based Framework for the MILP Problem in Robustness Verification of Neural Networks.

Neuron Importance Based Verification of Neural Networks Via Divide and Conquer

Training Verification-Friendly Neural Networks via Neuron Behavior Consistency

Efficient Complete Verification of Neural Networks Via Layerwised Splitting and Refinement

Expediting Neural Network Verification via Network Reduction

Tightened Convex Relaxations for Neural Network Robustness Certification

Towards Optimal Branching of Linear and Semidefinite Relaxations for Neural Network Robustness Certification