Abstract:Current state-of-the-art methods for human image customized synthesis typically require tedious training on large-scale datasets. In such cases, they are prone to overfitting and struggle to personalize individuals of unseen styles. Moreover, these methods extensively focus on single-concept human image synthesis and lack the flexibility needed for customizing individuals with multiple given concepts, thereby impeding their broader practical application. To this end, we propose MagicFace, a novel training-free method for universal-style human image personalized synthesis, enabling multi-concept customization by accurately integrating reference concept features into their latent generated region at the pixel level. Specifically, MagicFace introduces a coarse-to-fine generation pipeline, involving two sequential stages: semantic layout construction and concept feature injection. This is achieved by our Reference-aware Self-Attention (RSA) and Region-grouped Blend Attention (RBA) mechanisms. In the first stage, RSA enables the latent image to query features from all reference concepts simultaneously, extracting the overall semantic understanding to facilitate the initial semantic layout establishment. In the second stage, we employ an attention-based semantic segmentation method to pinpoint the latent generated regions of all concepts at each step. Following this, RBA divides the pixels of the latent image into semantic groups, with each group querying fine-grained features from the corresponding reference concept, which ensures precise attribute alignment and feature injection. Throughout the generation process, a weighted mask strategy is employed to ensure the model focuses more on the reference concepts. Extensive experiments demonstrate the superiority of MagicFace in both human-centric subject-to-image synthesis and multi-concept human image customization.

MAGIC: Mask-Guided Image Synthesis by Inverting a Quasi-Robust Classifier

MagicFace: Training-free Universal-Style Human Image Customized Synthesis

What to Hide from Your Students: Attention-Guided Masked Image Modeling

Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors

MAGIC: Mastering Physical Adversarial Generation in Context through Collaborative LLM Agents

Multi3D: 3D-Aware Multimodal Image Synthesis

MagicMix: Semantic Mixing with Diffusion Models

MixMask: Revisiting Masking Strategy for Siamese ConvNets

Plug-and-Play Controllable Generation for Discrete Masked Models

Disjoint Masking with Joint Distillation for Efficient Masked Image Modeling

Unlocking the Capabilities of Masked Generative Models for Image Synthesis via Self-Guidance

IMAGINE: Image Synthesis by Image-Guided Model Inversion

SemanticMIM: Marring Masked Image Modeling with Semantics Compression for General Visual Representation

SimMIM: A Simple Framework for Masked Image Modeling

Mask Conditional Synthetic Satellite Imagery

MaGIC: Multi-modality Guided Image Completion

MaskMamba: A Hybrid Mamba-Transformer Model for Masked Image Generation

Magicremover: Tuning-free Text-guided Image inpainting with Diffusion Models

Learning with Unmasked Tokens Drives Stronger Vision Learners

FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation Models

Mask-Guided Image Person Removal with Data Synthesis