Abstract:The recent advances in diffusion models (DMs) have revolutionized the generation of realistic and complex images. However, these models also introduce potential safety hazards, such as producing harmful content and infringing data copyrights. Despite the development of safety-driven unlearning techniques to counteract these challenges, doubts about their efficacy persist. To tackle this issue, we introduce an evaluation framework that leverages adversarial prompts to discern the trustworthiness of these safety-driven DMs after they have undergone the process of unlearning harmful concepts. Specifically, we investigated the adversarial robustness of DMs, assessed by adversarial prompts, when eliminating unwanted concepts, styles, and objects. We develop an effective and efficient adversarial prompt generation approach for DMs, termed UnlearnDiffAtk. This method capitalizes on the intrinsic classification abilities of DMs to simplify the creation of adversarial prompts, thereby eliminating the need for auxiliary classification or diffusion models. Through extensive benchmarking, we evaluate the robustness of widely-used safety-driven unlearned DMs (i.e., DMs after unlearning undesirable concepts, styles, or objects) across a variety of tasks. Our results demonstrate the effectiveness and efficiency merits of UnlearnDiffAtk over the state-of-the-art adversarial prompt generation method and reveal the lack of robustness of current safetydriven unlearning techniques when applied to DMs. Codes are available at <a class="link-external link-https" href="https://github.com/OPTML-Group/Diffusion-MU-Attack" rel="external noopener nofollow">this https URL</a>. WARNING: There exist AI generations that may be offensive in nature.

DisDet: Exploring Detectability of Backdoor Attack on Diffusion Models

Diff-Cleanse: Identifying and Mitigating Backdoor Attacks in Diffusion Models

How to Backdoor Diffusion Models?

Watch the Watcher! Backdoor Attacks on Security-Enhancing Diffusion Models

PureDiffusion: Using Backdoor to Counter Backdoor in Generative Diffusion Models

UIBDiffusion: Universal Imperceptible Backdoor Attack for Diffusion Models

Elijah: Eliminating Backdoors Injected in Diffusion Models via Distribution Shift

TERD: A Unified Framework for Safeguarding Diffusion Models Against Backdoors

UFID: A Unified Framework for Input-level Backdoor Detection on Diffusion Models

Invisible Backdoor Attacks on Diffusion Models

StealthDiffusion: Towards Evading Diffusion Forensic Detection through Diffusion Model

Attacks and Defenses for Generative Diffusion Models: A Comprehensive Survey

Data Forensics in Diffusion Models: A Systematic Analysis of Membership Privacy

Intriguing Properties of Diffusion Models: An Empirical Study of the Natural Attack Capability in Text-to-Image Generative Models

Defending Text-to-image Diffusion Models: Surprising Efficacy of Textual Perturbations Against Backdoor Attacks

Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation

Escaping Backdoor Attack Detection of Deep Learning

Exposing the Fake: Effective Diffusion-Generated Images Detection

To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now

Confidence Matters: Inspecting Backdoors in Deep Neural Networks via Distribution Transfer

Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoning