Mobile Robot Oriented Large-Scale Indoor Dataset for Dynamic Scene Understanding

Yifan Tang,Cong Tai,Fangxing Chen,Wanting Zhang,Tao Zhang,Xueping Liu,Yongjin Liu,Long Zeng

2024-07-01

Abstract:Most existing robotic datasets capture static scene data and thus are limited in evaluating robots' dynamic performance. To address this, we present a mobile robot oriented large-scale indoor dataset, denoted as THUD (Tsinghua University Dynamic) robotic dataset, for training and evaluating their dynamic scene understanding algorithms. Specifically, the THUD dataset construction is first detailed, including organization, acquisition, and annotation methods. It comprises both real-world and synthetic data, collected with a real robot platform and a physical simulation platform, respectively. Our current dataset includes 13 larges-scale dynamic scenarios, 90K image frames, 20M 2D/3D bounding boxes of static and dynamic objects, camera poses, and IMU. The dataset is still continuously expanding. Then, the performance of mainstream indoor scene understanding tasks, e.g. 3D object detection, semantic segmentation, and robot relocalization, is evaluated on our THUD dataset. These experiments reveal serious challenges for some robot scene understanding tasks in dynamic scenes. By sharing this dataset, we aim to foster and iterate new mobile robot algorithms quickly for robot actual working dynamic environment, i.e. complex crowded dynamic scenes.

Robotics

What problem does this paper attempt to address?

The paper attempts to address the issue that in existing mobile robot datasets, most datasets primarily capture data from static scenes, which poses limitations in evaluating the dynamic performance of robots. To overcome this limitation, the authors have constructed a large-scale indoor dataset for mobile robots (THUD) to train and evaluate algorithms for understanding dynamic scenes.

Specifically, the main contributions of the paper include:
1. **Dataset Construction**: A detailed introduction to the organization, collection, and annotation methods of the THUD dataset. This dataset includes both real-world and synthetic data, collected through real robot platforms and physical simulation platforms, respectively. The current version of the dataset includes 13 large-scale dynamic scenes, 90K image frames, 20M 2D/3D bounding boxes of static and dynamic objects, camera poses, and IMU data.
2. **Application Scenarios**: Evaluation of mainstream indoor scene understanding tasks (such as 3D object detection, semantic segmentation, and robot relocalization) on the THUD dataset. Experimental results reveal significant challenges faced by some robot scene understanding tasks in dynamic scenes.
3. **Dataset Expansion**: The dataset is continuously expanding to support more static and dynamic indoor mobile robot tasks, such as robot navigation in complex crowded dynamic scenes, target tracking, trajectory prediction, etc.

By sharing this dataset, the authors hope to promote and accelerate the development of new mobile robot algorithms to adapt to the dynamic environments in which robots actually operate.