PAD: Patch-Agnostic Defense against Adversarial Patch Attacks

Lihua Jing,Rui Wang,Wenqi Ren,Xin Dong,Cong Zou

2024-04-25

Abstract:Adversarial patch attacks present a significant threat to real-world object detectors due to their practical feasibility. Existing defense methods, which rely on attack data or prior knowledge, struggle to effectively address a wide range of adversarial patches. In this paper, we show two inherent characteristics of adversarial patches, semantic independence and spatial heterogeneity, independent of their appearance, shape, size, quantity, and location. Semantic independence indicates that adversarial patches operate autonomously within their semantic context, while spatial heterogeneity manifests as distinct image quality of the patch area that differs from original clean image due to the independent generation process. Based on these observations, we propose PAD, a novel adversarial patch localization and removal method that does not require prior knowledge or additional training. PAD offers patch-agnostic defense against various adversarial patches, compatible with any pre-trained object detectors. Our comprehensive digital and physical experiments involving diverse patch types, such as localized noise, printable, and naturalistic patches, exhibit notable improvements over state-of-the-art works. Our code is available at

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper focuses on the security threat posed by adversarial patch attacks on real-world object detectors. Existing defense methods rely on attack data or prior knowledge and are not effective against various adversarial patches. The authors discovered two inherent characteristics of adversarial patches: semantic independence and spatial heterogeneity. Semantic independence means that the patches operate independently in their semantic context, while spatial heterogeneity means that the patch regions differ in image quality from the original clean image due to the independent generation process.

Based on these observations, the paper proposes Patch-Agnostic Defense (PAD), an adversarial patch localization and removal method that does not require prior knowledge or additional training. PAD provides patch-agnostic defense against different types of patches and is compatible with any pre-trained object detector. Experiments demonstrate significant improvements of PAD on various patch types (such as local noise, printable, and natural appearance patches) in both digital and physical attack scenarios, compared to existing state-of-the-art works.

The main contributions of the paper are as follows:
1. Revealing two universal characteristics of adversarial patches and proposing a patch localization method based on mutual information and re-compression, which is independent of patch appearance, shape, size, location, and quantity.
2. Introducing PAD, a patch-agnostic defense method that does not require attack prior knowledge or additional training and is applicable to any object detector.
3. Demonstrating superior defense performance of PAD against different types of patch attacks in digital and physical experiments, surpassing current state-of-the-art methods.

In summary, the paper aims to address the effective defense against adversarial patch attacks by identifying and removing these attacks to enhance the security of object detectors.