Abstract:Neural networks have emerged as powerful tools across various domains, exhibiting remarkable empirical performance that motivated their widespread adoption in safety-critical applications, which, in turn, necessitates rigorous formal verification techniques to ensure their reliability and robustness. Tight bound propagation plays a crucial role in the formal verification process by providing precise bounds that can be used to formulate and verify properties, such as safety, robustness, and fairness. While state-of-the-art tools use linear and convex approximations to compute upper/lower bounds for each neuron's outputs, recent advances have shown that nonlinear approximations based on Bernstein polynomials lead to tighter bounds but suffer from scalability issues. To that end, this article introduces BERN-NN-IBF, a significant enhancement of the Bernstein-polynomial-based bound propagation algorithms. BERN-NN-IBF offers three main contributions: 1) a memory-efficient encoding of Bernstein polynomials to scale the bound propagation algorithms; 2) optimized tensor operations for the new polynomial encoding to maintain the integrity of the bounds while enhancing computational efficiency; and 3) tighter under-approximations of the ReLU activation function using quadratic polynomials tailored to minimize approximation errors. Through comprehensive testing, we demonstrate that BERN-NN-IBF achieves tighter bounds and higher computational efficiency compared to the original BERN-NN and state-of-the-art methods, including linear and convex programming used within the winner of the VNN-COMPETITION.

Twice-Pruning Aided WASD Neuronet of Bernoulli-Polynomial Type with Extension to Robust Classification

Class-Aware Pruning for Efficient Neural Networks

Wasd Algorithm With Pruning-While-Growing And Twice-Pruning Techniques For Multi-Input Euler Polynomial Neural Network

Sparse Bayesian Approach to Fast Learning Network for Multiclassification.

Weights and Structure Determination of 3-Input Bernoulli Polynomial Neural Net

Bernoulli Neural Network with Weights Directly Determined and with the Number of Hidden- Layer Neurons Automatically Determined

The 3-Input Euler Polynomial Neuronet (3IEPN) with Weights-and-structure-determination (WASD) Algorithm

Pruning-included weights and structure determination of 2-input neuronet using Chebyshev polynomials of Class 1

Growing-type Weights and Structure Determination of 2-Input Legendre Orthogonal Polynomial Neuronet.

Weights and Structure Determination of Multiple-Input Feed-Forward Neural Network Activated by Chebyshev Polynomials of Class 2 Via Cross-Validation

Cascading Model Based Back Propagation Neural Network in Enabling Precise Classification

A Gegenbauer Neural Network with Regularized Weights Direct Determination for Classification

Weights and Structure Determination of Pruning-While-Growing Type for 3-Input Power-Activation Feed-Forward Neuronet

RBF Neural Network Classifier with Weights and Structure Determination Method

A New Probabilistic Neural Network Model Based on Backpropagation Algorithm

BNN-DP: Robustness Certification of Bayesian Neural Networks via Dynamic Programming

Introducing the back-propagation into probabilistic neural network

BERN-NN-IBF: Enhancing Neural Network Bound Propagation Through Implicit Bernstein Form and Optimized Tensor Operations

MISO Multivariate Generalized Polynomials Neural Network and Its Weights-Direct-Determination

Weights and Structure Determination of Feed-Forward Two-Input Neural Network Activated by Chebyshev Polynomials of Class 2

A Lightweight Multi-Dendritic Pyramidal Neuron Model with Neural Plasticity on Image Recognition