Abstract:This paper explores methods for verifying the properties of Binary Neural Networks (BNNs), focusing on robustness against adversarial attacks. Despite their lower computational and memory needs, BNNs, like their full-precision counterparts, are also sensitive to input perturbations. Established methods for solving this problem are predominantly based on Satisfiability Modulo Theories and Mixed-Integer Linear Programming techniques, which are characterized by NP complexity and often face scalability issues. We introduce an alternative approach using Semidefinite Programming relaxations derived from sparse Polynomial Optimization. Our approach, compatible with continuous input space, not only mitigates numerical issues associated with floating-point calculations but also enhances verification scalability through the strategic use of tighter first-order semidefinite relaxations. We demonstrate the effectiveness of our method in verifying robustness against both $\|.\|_\infty$ and $\|.\|_2$-based adversarial attacks.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to solve the problem of robustness verification of binary neural networks (BNNs) in the face of adversarial attacks. Specifically, the paper focuses on the following points: 1. **Robustness verification**: Although BNNs have lower computational and memory requirements, they are as sensitive to input perturbations as full - precision neural networks. Therefore, verifying the robustness of BNNs is a crucial aspect in their design and deployment. 2. **Limitations of existing methods**: Currently, the methods used to solve this problem are mainly based on satisfiability modulo theory (SMT) and mixed - integer linear programming (MILP), and these methods usually face scalability problems. In addition, many formal verification frameworks do not fully utilize the bit - exact semantics of BNNs, and the existing BNN robustness verification methods are often limited by limited scalability. 3. **The proposed new method**: To solve the above problems, the author introduced a semi - definite programming (SDP) relaxation method based on sparse polynomial optimization. This method not only alleviates the numerical problems caused by floating - point operations but also enhances the scalability of verification by using a tighter first - order SDP relaxation. ### Specific content - **Problem background**: BNNs have received extensive attention due to their simple architecture and inherent advantages (such as reducing memory requirements and reducing computation time). However, quantized or binarized networks do not necessarily retain the properties of their real - valued precision counterparts, so it is crucial to verify their robustness. - **Deficiencies of existing methods**: - The SMT method encodes the BNN verification problem as Boolean formula satisfiability and uses off - the - shelf solvers to prove robustness or find counterexamples. - The MILP method regards robustness verification as an optimization problem, but these methods are exact verification methods, that is, they are reliable and complete, but difficult to handle in high - dimensional problems. - **Advantages of the new method**: - Utilize the semi - algebraic properties of symbolic activation functions, encode the BNN verification problem as a polynomial optimization problem (POP), and solve the first - order SDP relaxation to obtain a lower bound. - Improve the accuracy of the first - order SDP relaxation by adding redundant constraints (tautologies). - The method can quickly provide robustness certificates, more than 20 times faster than the traditional MILP algorithm. - **Verification effect**: This method performs well in verifying the robustness against ∥.∥∞ and ∥.∥2 - based adversarial attacks, especially being compatible in the continuous input space. ### Summary By introducing the SDP relaxation method based on sparse polynomial optimization, the paper effectively solves the problem of robustness verification of BNNs in the face of adversarial attacks. This method not only improves the accuracy of verification but also significantly improves the efficiency and scalability of verification.

Verifying Properties of Binary Neural Networks Using Sparse Polynomial Optimization

Improving Neural Network Verification through Spurious Region Guided Refinement.

Tight Verification of Probabilistic Robustness in Bayesian Neural Networks

Verifying Properties of Binarized Deep Neural Networks

Efficient Exact Verification of Binarized Neural Networks

Bound Tightening using Rolling-Horizon Decomposition for Neural Network Verification

Robustness Verification of Classification Deep Neural Networks Via Linear Programming.

Multi-Objective Linear Ensembles for Robust and Sparse Training of Few-Bit Neural Networks

BERN-NN-IBF: Enhancing Neural Network Bound Propagation Through Implicit Bernstein Form and Optimized Tensor Operations

A Dual Approach to Scalable Verification of Deep Networks

Automated Design of Linear Bounding Functions for Sigmoidal Nonlinearities in Neural Networks

Benchmarking Local Robustness of High-Accuracy Binary Neural Networks for Enhanced Traffic Sign Recognition

Precise Quantitative Analysis of Binarized Neural Networks: A BDD-based Approach

Robustness Verifcation in Neural Networks

Provably Bounding Neural Network Preimages

Scaling the Convex Barrier with Sparse Dual Algorithms

Efficient and Robust Mixed-Integer Optimization Methods for Training Binarized Deep Neural Networks

Quantitative Verification of Neural Networks And its Security Applications

DeepBern-Nets: Taming the Complexity of Certifying Neural Networks using Bernstein Polynomial Activations and Precise Bound Propagation

VNN: Verification-Friendly Neural Networks with Hard Robustness Guarantees

SecBNN: Efficient Secure Inference on Binary Neural Networks