Abstract:Recently, zero-shot methods like InstantID have revolutionized identity-preserving generation. Unlike multi-image finetuning approaches such as DreamBooth, these zero-shot methods leverage powerful facial encoders to extract identity information from a single portrait photo, enabling efficient identity-preserving generation through a single inference pass. However, this convenience introduces new threats to the facial identity protection. This paper aims to safeguard portrait photos from unauthorized encoder-based customization. We introduce IDProtector, an adversarial noise encoder that applies imperceptible adversarial noise to portrait photos in a single forward pass. Our approach offers universal protection for portraits against multiple state-of-the-art encoder-based methods, including InstantID, IP-Adapter, and PhotoMaker, while ensuring robustness to common image transformations such as JPEG compression, resizing, and affine transformations. Experiments across diverse portrait datasets and generative models reveal that IDProtector generalizes effectively to unseen data and even closed-source proprietary models.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to protect personal portrait photos from unauthorized encoder - based identity - preserving generation techniques. Specifically, with the development of zero - shot generation methods (such as InstantID, IP - Adapter, etc.), these methods can extract facial features from a single photo and generate realistic portraits, thus bringing potential risks of malicious use, such as generating malicious images, fake news or pornographic content. ### Core Problems of the Paper 1. **Risks of Identity - Preserving Generation**: - These zero - shot generation methods can efficiently generate realistic human face images from a single photo, but this also means that personal identity information may be misused. 2. **Insufficiency of Existing Protection Methods**: - Existing adversarial perturbation methods are mainly aimed at fine - tuning models (such as DreamBooth), and effective protection measures for encoder - based identity - preserving generation methods (such as InstantID and IP - Adapter) are not yet perfect. ### Solutions To solve the above problems, the author proposes a new method named IDProtector. This method protects images by adding imperceptible adversarial noise to portrait photos, so that these noises can effectively interfere with the extraction of identity features in multiple advanced encoder - based identity - preserving generation models, thereby preventing the generation of similar face images. ### Specific Objectives - **Universality**: Ensure that the protection method can resist multiple existing encoder - based identity - preserving generation methods. - **Efficiency**: Develop a feed - forward method to quickly and efficiently add protective noise to a large number of images. - **Robustness**: Ensure that the adversarial noise remains effective under common image transformations (such as JPEG compression, scaling, affine transformation, etc.). - **Imperceptibility**: Ensure that the added adversarial noise is imperceptible to the human visual system. ### Method Overview The main innovations of IDProtector include: - **ViT - based Noise Encoder**: Use Vision Transformer (ViT) to directly predict adversarial noise. - **Joint Learning Framework**: Design a composite loss function specifically for four mainstream encoder - based generation methods (InstantID, IP - Adapter, IP - Adapter Plus, PhotoMaker). - **Affine Transformation Data Augmentation**: By introducing small random noise in the face alignment stage, the robustness of the adversarial noise to common image transformations is enhanced. ### Experimental Results Experiments show that IDProtector performs excellently on multiple datasets such as CelebA and VGG Face. It not only significantly reduces the identity similarity (Identity Score Matching, ISM) between the generated image and the original image, but also is superior to existing methods in terms of computational efficiency and image quality impact. In conclusion, this paper aims to fill the research gap in the current literature on protecting personal identity information from encoder - based identity - preserving generation techniques by proposing IDProtector, and provides an efficient and robust solution.

IDProtector: An Adversarial Noise Encoder to Protect Against ID-Preserving Image Generation

"My Face, My Rules": Enabling Personalized Protection Against Unacceptable Face Editing.

FaceChain: A Playground for Identity-Preserving Portrait Generation

ID-Guard: A Universal Framework for Combating Facial Manipulation via Breaking Identification

Towards Privacy Protection by Generating Adversarial Identity Masks

InstantID: Zero-shot Identity-Preserving Generation in Seconds

Building an Invisible Shield for Your Portrait against Deepfakes

Adversarial Robust Safeguard for Evading Deep Facial Manipulation

InstantFamily: Masked Attention for Zero-shot Multi-ID Image Generation

Privacy preservation for image data: A GAN‐based method

IdentityDP: Differential Private Identification Protection for Face Images

Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing

High-Frequency Anti-DreamBooth: Robust Defense against Personalized Image Synthesis

Towards a Controllable and Reversible Privacy Protection System for Facial Images through Enhanced Multi-Factor Modifier Networks

Anti-Reference: Universal and Immediate Defense Against Reference-Based Generation

CLIP2Protect: Protecting Facial Privacy using Text-Guided Makeup via Adversarial Latent Search

Attack as Defense: Run-time Backdoor Implantation for Image Content Protection

Seeing is not Believing: An Identity Hider for Human Vision Privacy Protection

ID-Patch: Robust ID Association for Group Photo Personalization

Transferable Adversarial Facial Images for Privacy Protection

PID: Prompt-Independent Data Protection Against Latent Diffusion Models