Abstract:Image animation is to animate a still image of the object of interest using poses extracted from another video sequence. Through training on a large-scale video dataset, most existing approaches aim to explore disentangled appearance and pose representations of training frames. Then, the desired output with a specific appearance and pose can be synthesized via recombining learned representations. However, in some real-world applications, test images may lack the corresponding video ground-truth or follow a different distribution than the distribution of the training video frames (i.e., different domains), which largely limit the performance of existing methods. In this paper, we propose domain-independent pose representations that are compatible with and accessible by still images from a different domain. Specifically, we devise a two-stage self-supervised pose adaptation framework for general image animation tasks. A domain-independent pose adaptation generative adversarial network (DIPA-GAN) and a shuffle-patch generative adversarial network (Shuffle-patch GAN) are proposed to penalize the rationality of the synthesized frame's pose and appearance, respectively. Finally, experiments evaluated on various image animation tasks, which include same/cross-domain moving objects, facial expression transfer and human pose retargeting, demonstrate the superiority of the proposed framework over prior literature. Impact Statement—Image animation is a popular technology in video production. Benefiting from the rapid development of artificial intelligence (AI), recent image animation algorithms have been widely used in real-world applications, such as virtual AI news anchor, virtual try-on, and face swapping. However, most existing methods are designed for specific cases. To animate a new portrait, users are asked to collect hundreds of images of the same person and train a new model. The technology proposed in this paper overcomes these training limitations and generalizes image animations. In the challenging cross-domain facial expression transfer task, the user study demonstrated that our technology achieved more than 20% increase in animation success rate. The proposed technology could benefit users in a wide variety of industries including movie production, virtual reality, social media and online retail.

CoP: Chain-of-Pose for Image Animation in Large Pose Changes

Large Pose Friendly Face Reenactment using subtle motions

DisPose: Disentangling Pose Guidance for Controllable Human Image Animation

Model-based portrait video compression with spatial constraint and adaptive pose processing

COOP: Decoupling and Coupling of Whole-Body Grasping Pose Generation

Follow-Your-Pose v2: Multiple-Condition Guided Character Image Animation for Stable Pose Control

VividPose: Advancing Stable Video Diffusion for Realistic Human Image Animation

Self-Supervised Pose Adaptation for Cross-Domain Image Animation.

Image Comes Dancing With Collaborative Parsing-Flow Video Synthesis

Controllable Person Image Synthesis with Pose-Constrained Latent Diffusion

Do As I Do: Pose Guided Human Motion Copy

PISE-V: person image and video synthesis with decoupled GAN

Supervised Video-To-Video Synthesis For Single Human Pose Transfer

CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generation

Improved First-Order Motion Model of Image Animation with Enhanced Dense Motion and Repair Ability

CoMo: Controllable Motion Generation through Language Guided Pose Code Editing

Zero-shot High-fidelity and Pose-controllable Character Animation

Part-Preserving Pose Manipulation For Person Image Synthesis

Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videos

Synergizing Motion and Appearance: Multi-Scale Compensatory Codebooks for Talking Head Video Generation