Abstract:Generating high-fidelity human video with specified identities has attracted significant attention in the content generation community. However, existing techniques struggle to strike a balance between training efficiency and identity preservation, either requiring tedious case-by-case fine-tuning or usually missing identity details in the video generation process. In this study, we present \textbf{ID-Animator}, a zero-shot human-video generation approach that can perform personalized video generation given a single reference facial image without further training. ID-Animator inherits existing diffusion-based video generation backbones with a face adapter to encode the ID-relevant embeddings from learnable facial latent queries. To facilitate the extraction of identity information in video generation, we introduce an ID-oriented dataset construction pipeline that incorporates unified human attributes and action captioning techniques from a constructed facial image pool. Based on this pipeline, a random reference training strategy is further devised to precisely capture the ID-relevant embeddings with an ID-preserving loss, thus improving the fidelity and generalization capacity of our model for ID-specific video generation. Extensive experiments demonstrate the superiority of ID-Animator to generate personalized human videos over previous models. Moreover, our method is highly compatible with popular pre-trained T2V models like animatediff and various community backbone models, showing high extendability in real-world applications for video generation where identity preservation is highly desired. Our codes and checkpoints are released at <a class="link-external link-https" href="https://github.com/ID-Animator/ID-Animator" rel="external noopener nofollow">this https URL</a>.

Zero-Shot Face Swapping with De-identification Adversarial Learning

FaceSwapNet: Landmark Guided Many-to-Many Face Reenactment

Region-Aware Face Swapping

Deep Face Swapping via Cross-Identity Adversarial Training.

FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping

Designing One Unified Framework for High-Fidelity Face Reenactment and Swapping

Multi-modal Generative Adversarial Network for Zero-Shot Learning

Identity-Preserving Face Swapping via Dual Surrogate Generative Models

A Systematical Solution for Face De-identification

Meta-Transfer Networks for Zero-Shot Learning

MobileFaceSwap: A Lightweight Framework for Video Face Swapping

StableSwap: Stable Face Swapping in a Shared and Controllable Latent Space

Reinforced Disentanglement for Face Swapping Without Skip Connection

FSNet: An Identity-Aware Generative Model for Image-based Face Swapping

De-identification without losing faces

ID-Animator: Zero-Shot Identity-Preserving Human Video Generation

High-resolution Face Swapping via Latent Semantics Disentanglement

Learning Disentangled Representation for One-shot Progressive Face Swapping

On Face Segmentation, Face Swapping, and Face Perception

Enriching Facial Anti-Spoofing Datasets via an Effective Face Swapping Framework

Zero-Shot Visual Recognition Using Semantics-Preserving Adversarial Embedding Networks