Abstract:As a new programming paradigm, neural-network-based machine learning has expanded its application to many real-world problems. Due to the black-box nature of neural networks, verifying and explaining their behavior are becoming increasingly important, especially when they are deployed in safety-critical applications. Existing verification work mostly focuses on qualitative verification, which asks whether there exists an input (in a specified region) for a neural network such that a property (e.g., local robustness) is violated. However, in many practical applications, such an (adversarial) input almost surely exists, which makes a qualitative answer less meaningful. In this work, we study a more interesting yet more challenging problem, i.e., quantitative verification of neural networks, which asks how often a property is satisfied or violated. We target binarized neural networks (BNNs), the 1-bit quantization of general neural networks. BNNs have attracted increasing attention in deep learning recently, as they can drastically reduce memory storage and execution time with bit-wise operations, which is crucial in recourse-constrained scenarios, e.g., embedded devices for Internet of Things. Toward quantitative verification of BNNs, we propose a novel algorithmic approach for encoding BNNs as Binary Decision Diagrams (BDDs), a widely studied model in formal verification and knowledge representation. By exploiting the internal structure of the BNNs, our encoding translates the input-output relation of blocks in BNNs to cardinality constraints, which are then encoded by BDDs. Based on the new BDD encoding, we develop a quantitative verification framework for BNNs where precise and comprehensive analysis of BNNs can be performed. To improve the scalability of BDD encoding, we also investigate parallelization strategies at various levels. We demonstrate applications of our framework by providing quantitative robustness verification and interpretability for BNNs. An extensive experimental evaluation confirms the effectiveness and efficiency of our approach.

Taming Binarized Neural Networks and Mixed-Integer Programs

Efficient and Robust Mixed-Integer Optimization Methods for Training Binarized Deep Neural Networks

Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1

Hyperbolic Binary Neural Network

Training Restricted Boltzmann Machines with Binary Synapses Using the Bayesian Learning Rule

Understanding Neural Network Binarization with Forward and Backward Proximal Quantizers

Network Binarization via Contrastive Learning

Loss-aware Binarization of Deep Networks.

AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets

An adiabatic method to train binarized artificial neural networks

Learning to Solve Bilevel Programs with Binary Tender

Training Experimentally Robust and Interpretable Binarized Regression Models Using Mixed-Integer Programming

TP-ADMM: An Efficient Two-Stage Framework for Training Binary Neural Networks

Adjustable Bounded Rectifiers: Towards Deep Binary Representations

A foundation for exact binarized morphological neural networks

Precise Quantitative Analysis of Binarized Neural Networks: A BDD-based Approach

BDD4BNN: A BDD-Based Quantitative Analysis Framework for Binarized Neural Networks

Progressive Stochastic Binarization of Deep Networks

Gradient Corrected Approximation for Binary Neural Networks.

Binarizing by Classification: is Soft Function Really Necessary?

Recurrent Bilinear Optimization for Binary Neural Networks