Abstract:The non-consensual exploitation of facial manipulation has emerged as a pressing societal concern. In tandem with the identification of such fake content, recent research endeavors have advocated countering manipulation techniques through proactive interventions, specifically the incorporation of adversarial noise to impede the manipulation in advance. Nevertheless, with insufficient consideration of robustness, we show that current methods falter in providing protection after simple perturbations, e.g., blur. In addition, traditional optimization-based methods face limitations in scalability as they struggle to accommodate the substantial expansion of data volume, a consequence of the time-intensive iterative pipeline. To solve these challenges, we propose a learning-based model, Adversarial Robust Safeguard (ARS), to generate desirable protection noise in a single forward process, concurrently exhibiting a heightened resistance against prevalent perturbations. Specifically, our method involves a two-way protection design, characterized by a basic protection component responsible for generating efficacious noise features, coupled with robust protection for further enhancement. In robust protection, we first fuse image features with spatially duplicated noise embedding, thereby accounting for inherent information redundancy. Subsequently, a combination comprising a differentiable perturbation module and an adversarial network is devised to simulate potential information degradation during the training process. To evaluate it, we conduct experiments on four manipulation methods and compare recent works comprehensively. The results of our method exhibit good visual effects with pronounced robustness against varied perturbations at different levels.

Nearly Zero-Cost Protection Against Mimicry by Personalized Diffusion Models

Imperceptible Protection against Style Imitation from Diffusion Models

Toward effective protection against diffusion based mimicry through score distillation

Toward Robust Imperceptible Perturbation against Unauthorized Text-to-image Diffusion-based Synthesis

Real-time Identity Defenses against Malicious Personalization of Diffusion Models

Disrupting Style Mimicry Attacks on Video Imagery

IMPRESS: Evaluating the Resilience of Imperceptible Perturbations Against Unauthorized Data Usage in Diffusion-Based Generative AI

Pixel Is Not A Barrier: An Effective Evasion Attack for Pixel-Domain Diffusion Models

Perturbing Attention Gives You More Bang for the Buck: Subtle Imaging Perturbations That Efficiently Fool Customized Diffusion Models

DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing

Prompt-Agnostic Adversarial Perturbation for Customized Diffusion Models

Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation

Raising the Cost of Malicious AI-Powered Image Editing

Privacy Protection in Personalized Diffusion Models via Targeted Cross-Attention Adversarial Attack

Attack as Defense: Run-time Backdoor Implantation for Image Content Protection

MetaCloak: Preventing Unauthorized Subject-driven Text-to-image Diffusion-based Synthesis via Meta-learning

Can Protective Perturbation Safeguard Personal Data from Being Exploited by Stable Diffusion?

Rethinking and Defending Protective Perturbation in Personalized Diffusion Models

Adversarial Robust Safeguard for Evading Deep Facial Manipulation

Only My Model On My Data: A Privacy Preserving Approach Protecting one Model and Deceiving Unauthorized Black-Box Models

Anti-DreamBooth: Protecting users from personalized text-to-image synthesis