Abstract:Autonomous driving has attracted tremendous attention especially in the past few years. The key techniques for a self-driving car include solving tasks like 3D map construction, self-localization, parsing the driving road and understanding objects, which enable vehicles to reason and act. However, large scale data set for training and system evaluation is still a bottleneck for developing robust perception models. In this paper, we present the ApolloScape dataset [1] and its applications for autonomous driving. Compared with existing public datasets from real scenes, e.g. KITTI [2] or Cityscapes [3], ApolloScape contains much large and richer labelling including holistic semantic dense point cloud for each site, stereo, per-pixel semantic labelling, lanemark labelling, instance segmentation, 3D car instance, high accurate location for every frame in various driving videos from multiple sites, cities and daytimes. For each task, it contains at lease 15x larger amount of images than SOTA datasets. To label such a complete dataset, we develop various tools and algorithms specified for each task to accelerate the labelling process, such as 3D-2D segment labeling tools, active labelling in videos etc. Depend on ApolloScape, we are able to develop algorithms jointly consider the learning and inference of multiple tasks. In this paper, we provide a sensor fusion scheme integrating camera videos, consumer-grade motion sensors (GPS/IMU), and a 3D semantic map in order to achieve robust self-localization and semantic segmentation for autonomous driving. We show that practically, sensor fusion and joint learning of multiple tasks are beneficial to achieve a more robust and accurate system. We expect our dataset and proposed relevant algorithms can support and motivate researchers for further development of multi-sensor fusion and multi-task learning in the field of computer vision.

OmniHD-Scenes: A Next-Generation Multimodal Dataset for Autonomous Driving

nuScenes: A Multimodal Dataset for Autonomous Driving

One Million Scenes for Autonomous Driving: ONCE Dataset.

OpenMPD: An Open Multimodal Perception Dataset for Autonomous Driving

Zenseact Open Dataset: A large-scale and diverse multimodal dataset for autonomous driving

Scalability in Perception for Autonomous Driving: Waymo Open Dataset.

Dual Radar: A Multi-modal Dataset with Dual 4D Radar for Autonomous Driving

PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving

Dual Radar: A Multi-modal Dataset with Dual 4D Radar for Autononous Driving

A system of vision sensor based deep neural networks for complex driving scene analysis in support of crash risk assessment and prevention

AmodalSynthDrive: A Synthetic Amodal Perception Dataset for Autonomous Driving

The ApolloScape Open Dataset for Autonomous Driving and its Application

RoScenes: A Large-scale Multi-view 3D Dataset for Roadside Perception

SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving

aiMotive Dataset: A Multimodal Dataset for Robust Autonomous Driving with Long-Range Perception

The ADUULM-360 Dataset -- A Multi-Modal Dataset for Depth Estimation in Adverse Weather

UniScene: Multi-Camera Unified Pre-training via 3D Scene Reconstruction for Autonomous Driving

WaterScenes: A Multi-Task 4D Radar-Camera Fusion Dataset and Benchmarks for Autonomous Driving on Water Surfaces

EnvoDat: A Large-Scale Multisensory Dataset for Robotic Spatial Awareness and Semantic Reasoning in Heterogeneous Environments