Learning Generalizable Human Motion Generator with Reinforcement Learning

Yunyao Mao,Xiaoyang Liu,Wengang Zhou,Zhenbo Lu,Houqiang Li

2024-05-24

Abstract:Text-driven human motion generation, as one of the vital tasks in computer-aided content creation, has recently attracted increasing attention. While pioneering research has largely focused on improving numerical performance metrics on given datasets, practical applications reveal a common challenge: existing methods often overfit specific motion expressions in the training data, hindering their ability to generalize to novel descriptions like unseen combinations of motions. This limitation restricts their broader applicability. We argue that the aforementioned problem primarily arises from the scarcity of available motion-text pairs, given the many-to-many nature of text-driven motion generation. To tackle this problem, we formulate text-to-motion generation as a Markov decision process and present \textbf{InstructMotion}, which incorporate the trail and error paradigm in reinforcement learning for generalizable human motion generation. Leveraging contrastive pre-trained text and motion encoders, we delve into optimizing reward design to enable InstructMotion to operate effectively on both paired data, enhancing global semantic level text-motion alignment, and synthetic text-only data, facilitating better generalization to novel prompts without the need for ground-truth motion supervision. Extensive experiments on prevalent benchmarks and also our synthesized unpaired dataset demonstrate that the proposed InstructMotion achieves outstanding performance both quantitatively and qualitatively.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The main focus of this paper is to address the lack of generalization ability in existing text-driven human motion generation methods when faced with new descriptions. Existing methods often overfit specific motion expressions in the training data, limiting their ability to generate new combinations of actions. The authors believe that the problem lies in the scarcity of available motion-text paired data, given the multi-to-multi nature of text-driven motion generation tasks, which require a large number of training samples.

To solve this problem, the paper proposes a reinforcement learning-based method called InstructMotion. It treats text-to-motion generation as a Markov decision process and uses the trial-and-error mechanism of reinforcement learning to improve the generalization of human motion generation. By comparing pre-trained text and motion encoders, a reward model is designed that can optimize global semantic alignment between motion sequences and text descriptions without real motion supervision, promoting better generalization on unseen text descriptions.

Experiments on popular benchmark and synthetic unpaired datasets demonstrate that InstructMotion outperforms existing methods both quantitatively and qualitatively, particularly in handling novel motion combination descriptions, showing significant performance improvement. Additionally, the paper explores reward design, novel motion description generation methods, and reinforcement learning training processes.

In summary, the paper attempts to improve the generalization ability of text-driven human motion generation models to new descriptions, enabling wider applications.