Abstract:Transferring human motion from a source to a target person poses great potential in computer vision and graphics applications. A crucial step is to manipulate sequential future motion while retaining the appearance characteristic. Previous work has either relied on crafted 3D human models or trained a separate model specifically for each target person, which is not scalable in practice. This work studies a more general setting, in which we aim to learn a single model to parsimoniously transfer motion from a source video to any target person given only one image of the person, named as Collaborative Parsing-Flow Network (CPF-Net). The paucity of information regarding the target person makes the task particularly challenging to faithfully preserve the appearance in varying designated poses. To address this issue, CPF-Net integrates the structured human parsing and appearance flow to guide the realistic foreground synthesis which is merged into the background by a spatio-temporal fusion module. In particular, CPF-Net decouples the problem into stages of human parsing sequence generation, foreground sequence generation and final video generation. The human parsing generation stage captures both the pose and the body structure of the target. The appearance flow is beneficial to keep details in synthesized frames. The integration of human parsing and appearance flow effectively guides the generation of video frames with realistic appearance. Finally, the dedicated designed fusion network ensure the temporal coherence. We further collect a large set of human dancing videos to push forward this research field. Both quantitative and qualitative results show our method substantially improves over previous approaches and is able to generate appealing and photo-realistic target videos given any input person image. All source code and dataset will be released at https://github.com/xiezhy6/CPF-Net.

WaveIPT: Joint Attention and Flow Alignment in the Wavelet Domain for Pose Transfer

FreqHPT: Frequency-aware Attention and Flow Fusion for Human Pose Transfer.

Dense Intrinsic Appearance Flow for Human Pose Transfer

Towards Fine-Grained Human Pose Transfer With Detail Replenishing Network

FDA-GAN: Flow-based Dual Attention GAN for Human Pose Transfer

HairFIT: Pose-Invariant Hairstyle Transfer via Flow-based Hair Alignment and Semantic-Region-Aware Inpainting

PoseFlow: A Deep Motion Representation for Understanding Human Behaviors in Videos

A Cascaded Inception of Inception Network with Attention Modulated Feature Fusion for Human Pose Estimation

One-shot Human Motion Transfer via Occlusion-Robust Flow Prediction and Neural Texturing

PCFN: Progressive Cross-Modal Fusion Network for Human Pose Transfer

TSwinPose: Enhanced monocular 3D human pose estimation with JointFlow

Haar-wavelet based texture inpainting for human pose transfer

Combining Attention with Flow for Person Image Synthesis

Human Pose Estimation Via Dynamic Information Transfer

Hierarchy Flow For High-Fidelity Image-to-Image Translation

Image Comes Dancing With Collaborative Parsing-Flow Video Synthesis

Flowpose: Conditional Normalizing Flows for 3D Human Pose and Shape Estimation from Monocular Videos

C2F-FWN: Coarse-to-Fine Flow Warping Network for Spatial-Temporal Consistent Motion Transfer

Flow-guided attention deformation for person image generation

Collecting The Puzzle Pieces: Disentangled Self-Driven Human Pose Transfer by Permuting Textures

Joint-Motion Mutual Learning for Pose Estimation in Videos