Abstract:We introduce a novel method for synthesizing dance motions that follow the emotions and contents of a piece of music. Our method employs a learning-based approach to model the music to motion mapping relationship embodied in example dance motions along with those motions' accompanying background music. A key step in our method is to train a music to motion matching quality rating function through learning the music to motion mapping relationship exhibited in synchronized music and dance motion data, which were captured from professional human dance performance. To generate an optimal sequence of dance motion segments to match with a piece of music, we introduce a constraint-based dynamic programming procedure. This procedure considers both music to motion matching quality and visual smoothness of a resultant dance motion sequence. We also introduce a two-way evaluation strategy, coupled with a GPU-based implementation, through which we can execute the dynamic programming process in parallel, resulting in significant speedup. To evaluate the effectiveness of our method, we quantitatively compare the dance motions synthesized by our method with motion synthesis results by several peer methods using the motions captured from professional human dancers' performance as the gold standard. We also conducted several medium-scale user studies to explore how perceptually our dance motion synthesis method can outperform existing methods in synthesizing dance motions to match with a piece of music. These user studies produced very positive results on our music-driven dance motion synthesis experiments for several Asian dance genres, confirming the advantages of our method.

Research on motion editing for chime bell choreography

Motion Editing Based Dance Arrangement Assisting Technology

Implementation of a notation-based motion choreography system

Mocap data editing via movement notations

ChoreoMaster: choreography-oriented music-driven dance synthesis

A Framework of Mocap Data Editing with Labanotation

The Exploring of the Motion Capture Technologyin Bianzhong Choreography

Research on the Application of Information Fusion Technology in the Choreography of Hezhen Dance

Example-Based Automatic Music-Driven Conventional Dance Motion Synthesis

Exploring AI-assisted Ideation and Prototyping for Choreography

Exploring of the Motion Compilation System in Bianzhong Choreography

Data-Driven Computer Choreography Based on Kinect and 3D Technology

A New Motion Editing Algorithm Based On Motion Capture Data

ChoreoNet: Towards Music to Dance Synthesis with Choreographic Action Unit

Robotic Choreography Inspired by the Method of Human Dance Creation

Plan2Dance: Planning Based Choreographing from Music

Music-Driven emotion model applied in digitalized dance performance of sacrificial ceremony for confucius

A Knowledge-Based Intelligent System for Distributed and Collaborative Choreography.

Human motion tracking and system design for dance choreography teaching based on deep learning

Choreographing the Digital Canvas: A Machine Learning Approach to Artistic Performance