Abstract:Although deep learning models have achieved unprecedented success, their vulnerabilities towards adversarial attacks have attracted increasing attention, especially when deployed in security-critical domains. To address the challenge, numerous defense strategies, including reactive and proactive ones, have been proposed for robustness improvement. From the perspective of image feature space, some of them cannot reach satisfying results due to the shift of features. Besides, features learned by models are not directly related to classification results. Different from them, We consider defense method essentially from model inside and investigated the neuron behaviors before and after attacks. We observed that attacks mislead the model by dramatically changing the neurons that contribute most and least to the correct label. Motivated by it, we introduce the concept of neuron influence and further divide neurons into front, middle and tail part. Based on it, we propose neuron-level inverse perturbation(NIP), the first neuron-level reactive defense method against adversarial attacks. By strengthening front neurons and weakening those in the tail part, NIP can eliminate nearly all adversarial perturbations while still maintaining high benign accuracy. Besides, it can cope with different sizes of perturbations via adaptivity, especially larger ones. Comprehensive experiments conducted on three datasets and six models show that NIP outperforms the state-of-the-art baselines against eleven adversarial attacks. We further provide interpretable proofs via neuron activation and visualization for better understanding. Impact Statement—Deep learning has attracted tremendous attentions in many fields but some studies have proved that they are vulnerable to adversarial attacks. Meanwhile, defense methods against them are developed as well. Some solutions via simple image transformations are easy to implement but may be compromised in the event of larger perturbations. Methods based on feature space are recently proposed, by projecting or mappings the distribution of adversarial examples back to that of benign examples. But they may fail due to the shift of feature during operations on adversarial examples. Besides, feature distribution doesn’t directly relate to classifications. We propose NIP, a neuron-level defense against generic adversarial attack, which This research was supported by the National Natural Science Foundation of China under Grant No. 62072406, the Natural Science Foundation of Zhejiang Provincial under Grant No. LY19F020025. R. Chen is with the College of Information Engineering at Zhejiang University of Technology, Hangzhou 310007, China. (e-mail: 2112003149@zjut.edu.cn). H. Jin is with the College of Information Engineering at Zhejiang University of Technology, Hangzhou 310007, China. (e-mail: 2112003035@zjut.edu.cn). J. Chen is with the Institute of Cyberspace Security, College of Information Engineering, Zhejiang University of Technology, Hangzhou, 310023, China. (e-mail: chenjinyin@zjut.edu.cn) H. Zheng is with the College of Information Engineering at Zhejiang University of Technology, Hangzhou 310007, China. (e-mail: haibinzheng320@gmail.com). Y. Yue is with the Key Laboratory of Parallel and Distributed Computing, College of Computer, National University of Defense Technology, Changsha, 410000, China. (email: yuyue@nudt.edu.cn) S. Ji is with the College of Computer Science and Technology at Zhejiang University, Hangzhou 310007, China. (e-mail: sji@zju.edu.cn) bridge the neuron behaviors and correct classifications, from the perspective of model inside. By suppressing neurons exploited by attacks and enhancing class-relevant ones, it provides an attackagnostic, input-aware and more fine-grained solution for defense.

Defense against Adversarial Attacks with an Induced Class

Attack As Defense: Characterizing Adversarial Examples Using Robustness.

Adversarial for Good – Defending Training Data Privacy with Adversarial Attack Wisdom

A Universal Defense Strategy Against Adversarial Attacks Based on Attention-Guided

Attacking Adversarial Attacks as A Defense

Deep Defense: Training DNNs with Improved Adversarial Robustness

Evading Defenses to Transferable Adversarial Examples by Mitigating Attention Shift

Using Intuition from Empirical Properties to Simplify Adversarial Training Defense

DeepDefense: Training Deep Neural Networks with Improved Robustness.

Adversarial defense based on distribution transfer

Adversarial Example Defense via Perturbation Grading Strategy

Mitigating Adversarial Attacks for Deep Neural Networks by Input Deformation and Augmentation

Fight Perturbations with Perturbations: Defending Adversarial Attacks via Neuron Influence

Learning Defense Transformers for Counterattacking Adversarial Examples

NIP: Neuron-level Inverse Perturbation Against Adversarial Attacks.

Defense against adversarial attacks based on color space transformation

Square-Based Black-Box Adversarial Attack on Time Series Classification Using Simulated Annealing and Post-Processing-Based Defense

Are You Confident That You Have Successfully Generated Adversarial Examples?

Adversarial Examples Detection Beyond Image Space.

Meta Invariance Defense Towards Generalizable Robustness to Unknown Adversarial Attacks

Invisible Adversarial Attack Against Deep Neural Networks: an Adaptive Penalization Approach