Abstract:Quantized neural network (NN) with a reduced bit precision is an effective solution to reduces the computational and memory resource requirements and plays a vital role in machine learning. However, it is still challenging to avoid the significant accuracy degradation due to its numerical approximation and lower redundancy. In this paper, a novel robustness-aware 2-bit quantization scheme is proposed for NN base on binary NN and generative adversarial network(GAN), witch improves the performance by enriching the information of binary NN, efficiently extract the structural information and considering the robustness of the quantized NN. Specifically, using shift addition operation to replace the multiply-accumulate in the quantization process witch can effectively speed the NN. Meanwhile, a structural loss between the original NN and quantized NN is proposed to such that the structural information of data is preserved after quantization. The structural information learned from NN not only plays an important role in improving the performance but also allows for further fine tuning of the quantization network by applying the Lipschitz constraint to the structural loss. In addition, we also for the first time take the robustness of the quantized NN into consideration and propose a non-sensitive perturbation loss function by introducing an extraneous term of spectral norm. The experiments are conducted on CIFAR-10 and ImageNet datasets with popular NN( such as MoblieNetV2, SqueezeNet, ResNet20, etc). The experimental results show that the proposed algorithm is more competitive under 2-bit-precision than the state-of-the-art quantization methods. Meanwhile, the experimental results also demonstrate that the proposed method is robust under the FGSM adversarial samples attack.

An Integrated Approach to Produce Robust Models with High Efficiency

Optimizing Quantized Neural Networks in a Weak Curvature Manifold

Robust Training of Neural Networks at Arbitrary Precision and Sparsity

Sparsity Meets Robustness: Channel Pruning for the Feynman-Kac Formalism Principled Robust Deep Neural Nets

Robust Sparse Regularization: Simultaneously Optimizing Neural Network Robustness and Compactness

Double-Win Quant: Aggressively Winning Robustness of Quantized Deep Neural Networks Via Random Precision Training and Inference

Robustness-aware 2-bit quantization with real-time performance for neural network

EnResNet: ResNets Ensemble Via the Feynman-Kac Formalism for Adversarial Defense and Beyond

ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies

Error-Silenced Quantization: Bridging Robustness and Compactness

Sparsity Winning Twice: Better Robust Generalization from More Efficient Training

Improving Neural Network Efficiency Via Post-training Quantization with Adaptive Floating-Point

Robust Sparse Regularization: Defending Adversarial Attacks Via Regularized Sparse Network

Designing DNNs for a trade-off between robustness and processing performance in embedded devices

Adversarial Fine-tuning of Compressed Neural Networks for Joint Improvement of Robustness and Efficiency

Enhancing Adversarial Robustness in SNNs with Sparse Gradients

Defensive Quantization: When Efficiency Meets Robustness

Sparse DNNs with Improved Adversarial Robustness.

A Layer-wise Adversarial-aware Quantization Optimization for Improving Robustness

Benchmarking the Robustness of Quantized Models

Improving the Adversarial Robustness of Quantized Neural Networks Via Exploiting the Feature Diversity