Abstract:Diffusion models (DMs) have achieved remarkable success in text-to-image generation, but they also pose safety risks, such as the potential generation of harmful content and copyright violations. The techniques of machine unlearning, also known as concept erasing, have been developed to address these risks. However, these techniques remain vulnerable to adversarial prompt attacks, which can prompt DMs post-unlearning to regenerate undesired images containing concepts (such as nudity) meant to be erased. This work aims to enhance the robustness of concept erasing by integrating the principle of adversarial training (AT) into machine unlearning, resulting in the robust unlearning framework referred to as AdvUnlearn. However, achieving this effectively and efficiently is highly nontrivial. First, we find that a straightforward implementation of AT compromises DMs' image generation quality post-unlearning. To address this, we develop a utility-retaining regularization on an additional retain set, optimizing the trade-off between concept erasure robustness and model utility in AdvUnlearn. Moreover, we identify the text encoder as a more suitable module for robustification compared to UNet, ensuring unlearning effectiveness. And the acquired text encoder can serve as a plug-and-play robust unlearner for various DM types. Empirically, we perform extensive experiments to demonstrate the robustness advantage of AdvUnlearn across various DM unlearning scenarios, including the erasure of nudity, objects, and style concepts. In addition to robustness, AdvUnlearn also achieves a balanced tradeoff with model utility. To our knowledge, this is the first work to systematically explore robust DM unlearning through AT, setting it apart from existing methods that overlook robustness in concept erasing. Codes are available at: <a class="link-external link-https" href="https://github.com/OPTML-Group/AdvUnlearn" rel="external noopener nofollow">this https URL</a>

Erasing Undesirable Concepts in Diffusion Models with Adversarial Preservation

Removing Undesirable Concepts in Text-to-Image Diffusion Models with Learnable Prompts

Erasing Concepts from Diffusion Models

Erasing Undesirable Influence in Diffusion Models

Pruning for Robust Concept Erasing in Diffusion Models

All but One: Surgical Concept Erasing with Model Preservation in Text-to-Image Diffusion Models

Separable Multi-Concept Erasure from Diffusion Models

Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Models

Precise, Fast, and Low-cost Concept Erasure in Value Space: Orthogonal Complement Matters

Ablating Concepts in Text-to-Image Diffusion Models

Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models

Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradient

ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruning

Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Erasers

One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications

Unlearning or Concealment? A Critical Analysis and Evaluation Metrics for Unlearning in Diffusion Models

Probing Unlearned Diffusion Models: A Transferable Adversarial Attack Perspective

Unlearnable Examples for Diffusion Models: Protect Data from Unauthorized Exploitation

Unstable Unlearning: The Hidden Risk of Concept Resurgence in Diffusion Models

MACE: Mass Concept Erasure in Diffusion Models