GeRM: A Generalist Robotic Model with Mixture-of-experts for Quadruped Robot

Wenxuan Song,Han Zhao,Pengxiang Ding,Can Cui,Shangke Lyu,Yaning Fan,Donglin Wang

2024-04-09

Abstract:Multi-task robot learning holds significant importance in tackling diverse and complex scenarios. However, current approaches are hindered by performance issues and difficulties in collecting training datasets. In this paper, we propose GeRM (Generalist Robotic Model). We utilize offline reinforcement learning to optimize data utilization strategies to learn from both demonstrations and sub-optimal data, thus surpassing the limitations of human demonstrations. Thereafter, we employ a transformer-based VLA network to process multi-modal inputs and output actions. By introducing the Mixture-of-Experts structure, GeRM allows faster inference speed with higher whole model capacity, and thus resolves the issue of limited RL parameters, enhancing model performance in multi-task learning while controlling computational costs. Through a series of experiments, we demonstrate that GeRM outperforms other methods across all tasks, while also validating its efficiency in both training and inference processes. Additionally, we uncover its potential to acquire emergent skills. Additionally, we contribute the QUARD-Auto dataset, collected automatically to support our training approach and foster advancements in multi-task quadruped robot learning. This work presents a new paradigm for reducing the cost of collecting robot data and driving progress in the multi-task learning community. You can reach our project and video through the link:

Robotics,Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve

The paper primarily aims to address the following issues:

1. **Data Collection and Performance Issues in Multi-Task Robot Learning**:
- Current methods face difficulties in data collection and performance when dealing with diverse and complex scenarios. This paper proposes a General Robot Model (GeRM) that utilizes offline reinforcement learning to optimize data utilization, learning from demonstration data and suboptimal data, thereby surpassing the limitations of human demonstrations.

2. **Multi-Modal Input Processing and Action Output**:
- Utilizing a Transformer-based VLA network to process multi-modal inputs (such as text instructions and images) and generate action outputs. By introducing a Mixture-of-Experts structure, GeRM can achieve faster inference speeds while maintaining a high overall model capacity, addressing the limited parameters issue in reinforcement learning.

3. **Large-Scale Automatic Dataset Collection**:
- Proposes the QUARD-Auto dataset, which automatically collects a large amount of robot trajectory data, including both successful and failed data, to support training methods and promote the development of multi-task quadruped robot learning. This approach reduces the cost of data collection and advances the progress of the multi-task learning community.

In summary, the paper aims to solve the data collection challenges in multi-task robot learning, improve model performance through the GeRM model, and demonstrate its advantages in multi-task learning.