Abstract:Recent advances in text-to-motion generation using diffusion and autoregressive models have shown promising results. However, these models often suffer from a trade-off between real-time performance, high fidelity, and motion editability. To address this gap, we introduce MMM, a novel yet simple motion generation paradigm based on Masked Motion Model. MMM consists of two key components: (1) a motion tokenizer that transforms 3D human motion into a sequence of discrete tokens in latent space, and (2) a conditional masked motion transformer that learns to predict randomly masked motion tokens, conditioned on the pre-computed text tokens. By attending to motion and text tokens in all directions, MMM explicitly captures inherent dependency among motion tokens and semantic mapping between motion and text tokens. During inference, this allows parallel and iterative decoding of multiple motion tokens that are highly consistent with fine-grained text descriptions, therefore simultaneously achieving high-fidelity and high-speed motion generation. In addition, MMM has innate motion editability. By simply placing mask tokens in the place that needs editing, MMM automatically fills the gaps while guaranteeing smooth transitions between editing and non-editing parts. Extensive experiments on the HumanML3D and KIT-ML datasets demonstrate that MMM surpasses current leading methods in generating high-quality motion (evidenced by superior FID scores of 0.08 and 0.429), while offering advanced editing features such as body-part modification, motion in-betweening, and the synthesis of long motion sequences. In addition, MMM is two orders of magnitude faster on a single mid-range GPU than editable motion diffusion models. Our project page is available at \url{<a class="link-external link-https" href="https://exitudio.github.io/MMM-page" rel="external noopener nofollow">this https URL</a>}.

A Unified Masked Autoencoder with Patchified Skeletons for Motion Synthesis

OAW-GAN: Occlusion-Aware Warping GAN for Unified Human Video Synthesis

A Unified 3D Human Motion Synthesis Model Via Conditional Variational Auto-Encoder

Masked Motion Predictors Are Strong 3D Action Representation Learners

MoMask: Generative Masked Modeling of 3D Human Motions

Unimotion: Unifying 3D Human Motion Synthesis and Understanding

Dynamic Motion Synthesis: Masked Audio-Text Conditioned Spatio-Temporal Transformers

MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling

Skeleton2Humanoid: Animating Simulated Characters for Physically-plausible Motion In-betweening

Action-Conditioned 3D Human Motion Synthesis with Transformer VAE

HumanMAC: Masked Motion Completion for Human Motion Prediction

MMM: Generative Masked Motion Model

Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans

Social-MAE: Social Masked Autoencoder for Multi-person Motion Representation Learning

Multi-Resolution Generative Modeling of Human Motion from Limited Data

Auxiliary Tasks Benefit 3D Skeleton-based Human Motion Prediction

SkeletonMAE: Spatial-Temporal Masked Autoencoders for Self-supervised Skeleton Action Recognition.

MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpainting

Everything2Motion: Synchronizing Diverse Inputs Via a Unified Framework for Human Motion Synthesis

FreeMotion: MoCap-Free Human Motion Synthesis with Multimodal Large Language Models