Abstract:Quantized neural network (NN) with a reduced bit precision is an effective solution to reduces the computational and memory resource requirements and plays a vital role in machine learning. However, it is still challenging to avoid the significant accuracy degradation due to its numerical approximation and lower redundancy. In this paper, a novel robustness-aware 2-bit quantization scheme is proposed for NN base on binary NN and generative adversarial network(GAN), witch improves the performance by enriching the information of binary NN, efficiently extract the structural information and considering the robustness of the quantized NN. Specifically, using shift addition operation to replace the multiply-accumulate in the quantization process witch can effectively speed the NN. Meanwhile, a structural loss between the original NN and quantized NN is proposed to such that the structural information of data is preserved after quantization. The structural information learned from NN not only plays an important role in improving the performance but also allows for further fine tuning of the quantization network by applying the Lipschitz constraint to the structural loss. In addition, we also for the first time take the robustness of the quantized NN into consideration and propose a non-sensitive perturbation loss function by introducing an extraneous term of spectral norm. The experiments are conducted on CIFAR-10 and ImageNet datasets with popular NN( such as MoblieNetV2, SqueezeNet, ResNet20, etc). The experimental results show that the proposed algorithm is more competitive under 2-bit-precision than the state-of-the-art quantization methods. Meanwhile, the experimental results also demonstrate that the proposed method is robust under the FGSM adversarial samples attack.

Reliability Enhancement of Neural Networks via Neuron-Level Vulnerability Quantization

Enhancing Fault Resilience of QNNs by Selective Neuron Splitting

When Single Event Upset Meets Deep Neural Networks: Observations, Explorations, and Remedies

Improving Fault Tolerance for Reliable DNN Using Boundary-Aware Activation

Efficient Error-Tolerant Quantized Neural Network Accelerators

Correction to: Reliability Enhancement of Neural Networks via Neuron-Level Vulnerability Quantization

NV-DNN: Towards Fault-Tolerant DNN Systems with N-Version Programming

Robustness-aware 2-bit quantization with real-time performance for neural network

Reliability Assurance for Deep Neural Network Architectures Against Numerical Defects

DeepVigor: Vulnerability Value Ranges and Factors for DNNs' Reliability Assessment

A new method of software vulnerability detection based on a quantum neural network

DeepVigor+: Scalable and Accurate Semi-Analytical Fault Resilience Analysis for Deep Neural Network

Thales: Formulating and Estimating Architectural Vulnerability Factors for DNN Accelerators

Enabling Energy-Efficient and Reliable Neural Network via Neuron-Level Voltage Scaling

Exploring the Uncertainty Principle in Neural Networks Through Binary Classification

Quantifying the Impact of Single-Event Upsets on Neural Networks with Uncertainty Analysis

Improving Tolerance of Neural Networks Against Multi-Node Open Fault

Double-Win Quant: Aggressively Winning Robustness of Quantized Deep Neural Networks Via Random Precision Training and Inference

Recurrent quantum embedding neural network and its application in vulnerability detection

Resiliency of Deep Neural Networks under Quantization

Quantization Aware Attack: Enhancing Transferable Adversarial Attacks by Model Quantization