Abstract:Recently, digital humans for interpersonal interaction in virtual environments have gained significant attention. In this paper, we introduce a novel multi-dancer synthesis task called partner dancer generation, which involves synthesizing virtual human dancers capable of performing dance with users. The task aims to control the pose diversity between the lead dancer and the partner dancer. The core of this task is to ensure the controllable diversity of the generated partner dancer while maintaining temporal coordination with the lead dancer. This scenario varies from earlier research in generating dance motions driven by music, as our emphasis is on automatically designing partner dancer postures according to pre-defined diversity, the pose of lead dancer, as well as the accompanying tunes. To achieve this objective, we propose a three-stage framework called Dance-with-You (DanY). Initially, we employ a 3D Pose Collection stage to collect a wide range of basic dance poses as references for motion generation. Then, we introduce a hyper-parameter that coordinates the similarity between dancers by masking poses to prevent the generation of sequences that are over-diverse or consistent. To avoid the rigidity of movements, we design a Dance Pre-generated stage to pre-generate these masked poses instead of filling them with zeros. After that, a Dance Motion Transfer stage is adopted with leader sequences and music, in which a multi-conditional sampling formula is rewritten to transfer the pre-generated poses into a sequence with a partner style. In practice, to address the lack of multi-person datasets, we introduce AIST-M, a new dataset for partner dancer generation, which is publicly availiable. Comprehensive evaluations on our AIST-M dataset demonstrate that the proposed DanY can synthesize satisfactory partner dancer results with controllable diversity.

GroupDancer: Music to Multi-People Dance Synthesis with Style Collaboration

Music-Driven Group Choreography

DanceCamera3D: 3D Camera Movement Synthesis with Music and Dance

Dance with You: The Diversity Controllable Dancer Generation via Diffusion Models

Magic: Multi Art Genre Intelligent Choreography Dataset and Network for 3D Dance Generation

CoDancers: Music-Driven Coherent Group Dance Generation with Choreographic Unit

Robust Dancer: Long-term 3D Dance Synthesis Using Unpaired Data

DeepDance: Music-to-Dance Motion Choreography With Adversarial Learning

ChoreoNet: Towards Music to Dance Synthesis with Choreographic Action Unit

EnchantDance: Unveiling the Potential of Music-Driven Dance Movement

FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generation

Controllable Group Choreography using Contrastive Diffusion

ChoreoMaster: choreography-oriented music-driven dance synthesis

Mining and Applying Composition Knowledge of Dance Moves for Style-Concentrated Dance Generation

Dance Generation with Style Embedding: Learning and Transferring Latent Representations of Dance Styles

MDSC: Towards Evaluating the Style Consistency Between Music and Dance

Example-Based Automatic Music-Driven Conventional Dance Motion Synthesis

Music2Dance: DanceNet for Music-Driven Dance Generation