Abstract:The presence of adversarial examples poses a significant threat to deep learning models and their applications. Existing defense methods provide certain resilience against adversarial examples, but often suffer from decreased accuracy and generalization performance, making it challenging to achieve a trade-off between robustness and generalization. To address this, our paper interprets the adversarial example problem from the perspective of sample distribution and proposes a defense method based on distribution shift, leveraging the distribution transfer capability of a diffusion model for adversarial defense. The core idea is to exploit the discrepancy between normal and adversarial sample distributions to achieve adversarial defense using a pretrained diffusion model. Specifically, an adversarial sample undergoes a forward diffusion process, moving away from the source distribution, followed by a reverse process guided by the protected model (victim model) output to map it back to the normal distribution. Experimental evaluations on CIFAR10 and ImageNet30 datasets are conducted, comparing with adversarial training and input preprocessing methods. For infinite-norm attacks with 8/255 perturbation, accuracy rates of 78.1% and 83.5% are achieved, respectively. For 2-norm attacks with 128/255 perturbation, accuracy rates are 74.3% and 82.5%. Additional experiments considering perturbation amplitude, diffusion iterations, and adaptive attacks also validate the effectiveness of the proposed method. Results demonstrate that even when the attacker has knowledge of the defense, the proposed distribution-based method effectively withstands adversarial examples. It fills the gaps of traditional approaches, restoring high-quality original samples and showcasing superior performance in model robustness and generalization.

Adversarial Examples for Preventing Diffusion Models from Malicious Image Edition

Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examples

Raising the Cost of Malicious AI-Powered Image Editing

Mist: Towards Improved Adversarial Examples for Diffusion Models

Deceptive Diffusion: Generating Synthetic Adversarial Examples

DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing

AdvI2I: Adversarial Image Attack on Image-to-Image Diffusion models

Adversarial Examples are Misaligned in Diffusion Model Manifolds

The Stronger the Diffusion Model, the Easier the Backdoor: Data Poisoning to Induce Copyright Breaches Without Adjusting Finetuning Pipeline

Pixel is a Barrier: Diffusion Models Are More Adversarially Robust Than We Think

Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation

StealthDiffusion: Towards Evading Diffusion Forensic Detection through Diffusion Model

AdvDiff: Generating Unrestricted Adversarial Examples using Diffusion Models

Watermark-embedded Adversarial Examples for Copyright Protection against Diffusion Models

DiffProtect: Generate Adversarial Examples with Diffusion Models for Facial Privacy Protection

Adversarial defense based on distribution transfer

Diffusion Models for Imperceptible and Transferable Adversarial Attack

Pixel Is Not A Barrier: An Effective Evasion Attack for Pixel-Domain Diffusion Models

Perturbing Attention Gives You More Bang for the Buck: Subtle Imaging Perturbations That Efficiently Fool Customized Diffusion Models

Unveiling Universal Forensics of Diffusion Models with Adversarial Perturbations

IMPRESS: Evaluating the Resilience of Imperceptible Perturbations Against Unauthorized Data Usage in Diffusion-Based Generative AI