Abstract:Transferring human motion from a source to a target person poses great potential in computer vision and graphics applications. A crucial step is to manipulate sequential future motion while retaining the appearance characteristic. Previous work has either relied on crafted 3D human models or trained a separate model specifically for each target person, which is not scalable in practice. This work studies a more general setting, in which we aim to learn a single model to parsimoniously transfer motion from a source video to any target person given only one image of the person, named as Collaborative Parsing-Flow Network (CPF-Net). The paucity of information regarding the target person makes the task particularly challenging to faithfully preserve the appearance in varying designated poses. To address this issue, CPF-Net integrates the structured human parsing and appearance flow to guide the realistic foreground synthesis which is merged into the background by a spatio-temporal fusion module. In particular, CPF-Net decouples the problem into stages of human parsing sequence generation, foreground sequence generation and final video generation. The human parsing generation stage captures both the pose and the body structure of the target. The appearance flow is beneficial to keep details in synthesized frames. The integration of human parsing and appearance flow effectively guides the generation of video frames with realistic appearance. Finally, the dedicated designed fusion network ensure the temporal coherence. We further collect a large set of human dancing videos to push forward this research field. Both quantitative and qualitative results show our method substantially improves over previous approaches and is able to generate appealing and photo-realistic target videos given any input person image. All source code and dataset will be released at https://github.com/xiezhy6/CPF-Net.

Let's Dance: Learning From Online Dance Videos

Unsupervised 3D Pose Estimation for Hierarchical Dance Video Recognition

May the Dance be with You: Dance Generation Framework for Non-Humanoids

DeepDance: Music-to-Dance Motion Choreography With Adversarial Learning

Image Comes Dancing With Collaborative Parsing-Flow Video Synthesis

An Interactive Teaching System for Ethnic Minority Dance Based on Deep Learning

Human Action Recognition Using Deep Learning Methods.

Classification of dances using AlexNet, ResNet18 and SqueezeNet1_0

Dance Video Motion Recognition Based on Computer Vision and Image Processing

Nrityantar: Pose oblivious Indian classical dance sequence classification system

Dance Any Beat: Blending Beats with Visuals in Dance Video Generation

FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generation

Posture and sequence recognition for Bharatanatyam dance performances using machine learning approach

Learning to dance: A graph convolutional adversarial network to generate realistic dance motions from audio

Introducing the Simulated Flying Shapes and Simulated Planar Manipulator Datasets

Will You Ever Become Popular? Learning to Predict Virality of Dance Clips

Sensor based Dance Coherent Action Generation Model using Deep Learning Framework

Dance Art Scene Classification Based on Convolutional Neural Networks

Beat Based Realistic Dance Video Generation using Deep Learning

A New Residual Dense Network for Dance Action Recognition From Heterogeneous View Perception