InstantID: Zero-shot Identity-Preserving Generation in Seconds

Qixun Wang,Xu Bai,Haofan Wang,Zekui Qin,Anthony Chen,Huaxia Li,Xu Tang,Yao Hu

2024-02-03

Abstract:There has been significant progress in personalized image synthesis with methods such as Textual Inversion, DreamBooth, and LoRA. Yet, their real-world applicability is hindered by high storage demands, lengthy fine-tuning processes, and the need for multiple reference images. Conversely, existing ID embedding-based methods, while requiring only a single forward inference, face challenges: they either necessitate extensive fine-tuning across numerous model parameters, lack compatibility with community pre-trained models, or fail to maintain high face fidelity. Addressing these limitations, we introduce InstantID, a powerful diffusion model-based solution. Our plug-and-play module adeptly handles image personalization in various styles using just a single facial image, while ensuring high fidelity. To achieve this, we design a novel IdentityNet by imposing strong semantic and weak spatial conditions, integrating facial and landmark images with textual prompts to steer the image generation. InstantID demonstrates exceptional performance and efficiency, proving highly beneficial in real-world applications where identity preservation is paramount. Moreover, our work seamlessly integrates with popular pre-trained text-to-image diffusion models like SD1.5 and SDXL, serving as an adaptable plugin. Our codes and pre-trained checkpoints will be available at

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

The paper aims to address key challenges in personalized image synthesis, particularly the issue of identity preservation for faces. Specifically, the paper addresses the following points:

1. **High Storage Demand and Long Fine-Tuning Time**: Existing personalized image generation methods (such as Textual Inversion, DreamBooth, and LoRA) typically require a large amount of storage space and a long time for model fine-tuning.

2. **Need for Multiple Reference Images**: These methods often require multiple reference images to achieve better generation results.

3. **Limitations of Existing ID Embedding Methods**: Although some ID embedding-based methods can achieve personalized image generation with just one forward pass, they either require extensive model fine-tuning, are incompatible with community pre-trained models, or fail to maintain high facial fidelity.

To address the above issues, the authors propose a new method called InstantID, a diffusion model-based solution with the following features:

- **Instantaneity and Efficiency**: InstantID can generate personalized images using only one facial image within seconds while ensuring high-quality facial fidelity.

- **Compatibility and Plug-and-Play Design**: This method achieves this by constructing a lightweight adapter module that can be easily integrated with existing pre-trained models in the community without additional fine-tuning steps.

- **Robust Performance**: Even with just one reference image, InstantID can achieve or even surpass the performance levels of methods (such as LoRA) that require multiple reference images and additional training.

To achieve these goals, the paper introduces several key technical points, including the design of a new IdentityNet network for extracting strong semantic and weak spatial conditions from reference images, combining facial images, landmark images, and text prompts to guide the image generation process. Additionally, the paper details the training and inference strategies, as well as a series of experimental results, demonstrating that InstantID can maintain identity consistency while also retaining high editing capabilities and compatibility with existing control models.