Abstract:Diffusion models (DMs) have achieved remarkable success in text-to-image generation, but they also pose safety risks, such as the potential generation of harmful content and copyright violations. The techniques of machine unlearning, also known as concept erasing, have been developed to address these risks. However, these techniques remain vulnerable to adversarial prompt attacks, which can prompt DMs post-unlearning to regenerate undesired images containing concepts (such as nudity) meant to be erased. This work aims to enhance the robustness of concept erasing by integrating the principle of adversarial training (AT) into machine unlearning, resulting in the robust unlearning framework referred to as AdvUnlearn. However, achieving this effectively and efficiently is highly nontrivial. First, we find that a straightforward implementation of AT compromises DMs' image generation quality post-unlearning. To address this, we develop a utility-retaining regularization on an additional retain set, optimizing the trade-off between concept erasure robustness and model utility in AdvUnlearn. Moreover, we identify the text encoder as a more suitable module for robustification compared to UNet, ensuring unlearning effectiveness. And the acquired text encoder can serve as a plug-and-play robust unlearner for various DM types. Empirically, we perform extensive experiments to demonstrate the robustness advantage of AdvUnlearn across various DM unlearning scenarios, including the erasure of nudity, objects, and style concepts. In addition to robustness, AdvUnlearn also achieves a balanced tradeoff with model utility. To our knowledge, this is the first work to systematically explore robust DM unlearning through AT, setting it apart from existing methods that overlook robustness in concept erasing. Codes are available at: <a class="link-external link-https" href="https://github.com/OPTML-Group/AdvUnlearn" rel="external noopener nofollow">this https URL</a>

Dark Miner: Defend against undesired generation for text-to-image diffusion models

Toward Robust Imperceptible Perturbation against Unauthorized Text-to-image Diffusion-based Synthesis

Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Models

Circumventing Concept Erasure Methods For Text-to-Image Generative Models

Generating Natural Language Adversarial Examples on a Large Scale with Generative Models

MetaCloak: Preventing Unauthorized Subject-driven Text-to-image Diffusion-based Synthesis via Meta-learning

Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models

Degeneration-Tuning: Using Scrambled Grid shield Unwanted Concepts from Stable Diffusion

All but One: Surgical Concept Erasing with Model Preservation in Text-to-Image Diffusion Models

EIUP: A Training-Free Approach to Erase Non-Compliant Concepts Conditioned on Implicit Unsafe Prompts

DIAGNOSIS: Detecting Unauthorized Data Usages in Text-to-image Diffusion Models

Defending Text-to-image Diffusion Models: Surprising Efficacy of Textual Perturbations Against Backdoor Attacks

Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation

The Art of Deception: Black-box Attack Against Text-to-Image Diffusion Model

To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now

R.A.C.E.: Robust Adversarial Concept Erasure for Secure Text-to-Image Diffusion Model

Divide-and-Conquer Attack: Harnessing the Power of LLM to Bypass the Censorship of Text-to-Image Generation Model

Ablating Concepts in Text-to-Image Diffusion Models

SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Models

Discovering Failure Modes of Text-guided Diffusion Models via Adversarial Search