Abstract:The rapid development of deep learning has significantly transformed the ecology of the software engineering field. As new data continues to grow and evolve at an explosive rate, the challenge of iteratively updating software built on neural networks has become a critical issue. While the continuous learning paradigm enables networks to incorporate new data and update accordingly without losing previous memories, resulting in a batch of new networks as candidates for software updating, these approaches merely select from these networks by empirically testing their accuracy; they lack formal guarantees for such a batch of networks, especially in the presence of adversarial samples. Existing verification techniques, based on constraint solving, interval propagation, and linear approximation, provide formal guarantees but are designed to verify the properties of individual networks rather than a batch of networks. To address this issue, we analyze the batch verification problem corresponding to several non-traditional machine learning paradigms and further propose a framework named HOBAT (BATch verification for HOmogeneous structural neural networks) to enhance batch verification under reasonable assumptions about the representation of homogeneous structure neural networks, increasing scalability in practical applications. Our method involves abstracting the neurons at the same position in a batch of networks into a single neuron, followed by an iterative refinement process on the abstracted neuron to restore the precision until the desired properties for verification are met. Our method is orthogonal to boundary propagation verification on a single neural network. To assess our methodology, we integrate it with boundary propagation verification and observe significant improvements compared to the vanilla approach. Our experiments demonstrate the enormous potential for verifying large batches of networks in the era of big data.

Bound Tightening using Rolling-Horizon Decomposition for Neural Network Verification

A Dual Relaxation Method for Neural Network Verification

Probabilistic Verification of Neural Networks using Branch and Bound

A Dual Approach to Scalable Verification of Deep Networks

Neural Network Verification with Branch-and-Bound for General Nonlinearities

Quantitative Verification with Neural Networks

Provably Bounding Neural Network Preimages

Efficient verification of neural networks based on neuron branching and LP abstraction

General Cutting Planes for Bound-Propagation-Based Neural Network Verification

Tight Neural Network Verification via Semidefinite Relaxations and Linear Reformulations

Automated Design of Linear Bounding Functions for Sigmoidal Nonlinearities in Neural Networks

Tight Verification of Probabilistic Robustness in Bayesian Neural Networks

Verifying message-passing neural networks via topology-based bounds tightening

Harnessing Neuron Stability to Improve DNN Verification

Scaling the Convex Barrier with Sparse Dual Algorithms

BERN-NN-IBF: Enhancing Neural Network Bound Propagation Through Implicit Bernstein Form and Optimized Tensor Operations

Scalable Neural Network Verification with Branch-and-bound Inferred Cutting Planes

Verification of Geometric Robustness of Neural Networks via Piecewise Linear Approximation and Lipschitz Optimisation

HOBAT: Batch Verification for Homogeneous Structural Neural Networks

Expediting Neural Network Verification via Network Reduction

An RNN-Based Framework for the MILP Problem in Robustness Verification of Neural Networks.