Abstract:The ability to generate complex and realistic human body animations at scale, while following specific artistic constraints, has been a fundamental goal for the game and animation industry for decades. Popular techniques include key-framing, physics-based simulation, and database methods via motion graphs. Recently, motion generators based on deep learning have been introduced. Although these learning models can automatically generate highly intricate stylized motions of arbitrary length, they still lack user control. To this end, we introduce the problem of long-term inbetweening, which involves automatically synthesizing complex motions over a long time interval given very sparse keyframes by users. We identify a number of challenges related to this problem, including maintaining biomechanical and keyframe constraints, preserving natural motions, and designing the entire motion sequence holistically while considering all constraints. We introduce a biomechanically constrained generative adversarial network that performs long-term inbetweening of human motions, conditioned on keyframe constraints. This network uses a novel two-stage approach where it first predicts local motion in the form of joint angles, and then predicts global motion, i.e. the global path that the character follows. Since there are typically a number of possible motions that could satisfy the given user constraints, we also enable our network to generate a variety of outputs with a scheme that we call Motion DNA. This approach allows the user to manipulate and influence the output content by feeding seed motions (DNA) to the network. Trained with 79 classes of captured motion data, our network performs robustly on a variety of highly complex motion styles.

Multi-person/Group Interactive Video Generation

Multi-person/Group Interactive Video Generation

Two-in-One: Unified Multi-Person Interactive Motion Generation by Latent Diffusion Transformer

CoMA: Compositional Human Motion Generation with Multi-modal Agents

MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance

Towards Open Domain Text-Driven Synthesis of Multi-Person Motions

Co-Speech Gesture Video Generation via Motion-Decoupled Diffusion Model

Skeleton-Aided Articulated Motion Generation

Learning a Generative Model for Multi‐Step Human‐Object Interactions from Videos

AMG: Avatar Motion Guided Video Generation

Generative Tweening: Long-term Inbetweening of 3D Human Motions

ViMo: Generating Motions from Casual Videos

Action2video: Generating Videos of Human 3D Actions

Multi-Frame Content Integration with a Spatio-Temporal Attention Mechanism for Person Video Motion Transfer

InterGen: Diffusion-Based Multi-human Motion Generation Under Complex Interactions

Multi-Modal Driven Pose-Controllable Talking Head Generation

Text-guided 3D Human Motion Generation with Keyframe-based Parallel Skip Transformer

Motion-Agent: A Conversational Framework for Human Motion Generation with LLMs

Fg-T2M: Fine-Grained Text-Driven Human Motion Generation Via Diffusion Model

Composable Semi parametric Modelling for Long range Motion Generation

GUESS:GradUally Enriching SyntheSis for Text-Driven Human Motion Generation