Abstract:Human motion copy is an intriguing yet challenging task in artificial intelligence and computer vision, which strives to generate a fake video of a target person performing the motion of a source person. The problem is inherently challenging due to the subtle human-body texture details to be generated and the temporal consistency to be considered. Existing approaches typically adopt a conventional GAN with an L1 or L2 loss to produce the target fake video, which intrinsically necessitates a large number of training samples that are challenging to acquire. Meanwhile, current methods still have difficulties in attaining realistic image details and temporal consistency, which unfortunately can be easily perceived by human observers. Motivated by this, we try to tackle the issues from three aspects: (1) We constrain pose-to-appearance generation with a perceptual loss and a theoretically motivated Gromov-Wasserstein loss to bridge the gap between pose and appearance. (2) We present an episodic memory module in the pose-to-appearance generation to propel continuous learning that helps the model learn from its past poor generations. We also utilize geometrical cues of the face to optimize facial details and refine each key body part with a dedicated local GAN. (3) We advocate generating the foreground in a sequence-to-sequence manner rather than a single-frame manner, explicitly enforcing temporal inconsistency. Empirical results on five datasets, iPER, ComplexMotion, SoloDance, Fish, and Mouse datasets, demonstrate that our method is capable of generating realistic target videos while precisely copying motion from a source video. Our method significantly outperforms state-of-the-art approaches and gains 7.2% and 12.4% improvements in PSNR and FID respectively.

Mona Lisa alive

Video Tracked Facial Expression Animation

Performance-Driven Animation of Hand-Drawn Cartoon Faces

Based on Human-like Approach

Quick Customization of Particular Human Face and Muscle-Driven Expression Animation

Facial Expression Animation Based on Physical Model

Novel 3D-Aware Composition Images Synthesis for Object Display with Diffusion Model.

Animating Cartoon Faces by Multi-View Drawings

A hybrid approach to animating the murals with Dunhuang style

SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video

Interactive Facial-Geometric-Feature Animation For Generating Expressions Of Novel Faces

Personalized Multi-View Face Animation with Lifelike Textures

Mimicing 3D transformations of emotional stylised animation with minimal 2D input

Human-Inspired Facial Sketch Synthesis with Dynamic Adaptation

AniArtAvatar: Animatable 3D Art Avatar from a Single Image

Do as I Do: Pose Guided Human Motion Copy

Real-time Facial Animation with Image-Based Dynamic Avatars.

Hand Avatar: Free-Pose Hand Animation and Rendering from Monocular Video

Alive Caricature from 2D to 3D

Synthesizing Moving People with 3D Control