Abstract:Autonomous driving vehicles rely on sensors for the robust perception of their surroundings. Such vehicles are equipped with multiple perceptive sensors with a high level of redundancy to ensure safety and reliability in any driving condition. However, multi-sensor, such as camera, LiDAR, and radar systems raise requirements related to sensor calibration and synchronization, which are the fundamental blocks of any autonomous system. On the other hand, sensor fusion and integration have become important aspects of autonomous driving research and directly determine the efficiency and accuracy of advanced functions such as object detection and path planning. Classical model-based estimation and data-driven models are two mainstream approaches to achieving such integration. Most recent research is shifting to the latter, showing high robustness in real-world applications but requiring large quantities of data to be collected, synchronized, and properly categorized. However, there are two major research gaps in existing works: (i) they lack fusion (and synchronization) of multi-sensors, camera, LiDAR and radar; and (ii) generic scalable, and user-friendly end-to-end implementation. To generalize the implementation of the multi-sensor perceptive system, we introduce an end-to-end generic sensor dataset collection framework that includes both hardware deploying solutions and sensor fusion algorithms. The framework prototype integrates a diverse set of sensors, such as camera, LiDAR, and radar. Furthermore, we present a universal toolbox to calibrate and synchronize three types of sensors based on their characteristics. The framework also includes the fusion algorithms, which utilize the merits of three sensors, namely, camera, LiDAR, and radar, and fuse their sensory information in a manner that is helpful for object detection and tracking research. The generality of this framework makes it applicable in any robotic or autonomous applications and suitable for quick and large-scale practical deployment.

ECMD: An Event-Centric Multisensory Driving Dataset for SLAM

DIDLM:A Comprehensive Multi-Sensor Dataset with Infrared Cameras, Depth Cameras, LiDAR, and 4D Millimeter-Wave Radar in Challenging Scenarios for 3D Mapping

S3E: A Large-scale Multimodal Dataset for Collaborative SLAM

DSEC: A Stereo Event Camera Dataset for Driving Scenarios

CID-SIMS: Complex indoor dataset with semantic information and multi-sensor data from a ground wheeled robot viewpoint

MCD: Diverse Large-Scale Multi-Campus Dataset for Robot Perception

OpenMPD: An Open Multimodal Perception Dataset for Autonomous Driving

CARLA-Loc: Synthetic SLAM Dataset with Full-stack Sensor Setup in Challenging Weather and Dynamic Environments

CoSEC: A Coaxial Stereo Event Camera Dataset for Autonomous Driving

VECtor: A Versatile Event-Centric Benchmark for Multi-Sensor SLAM

OmniHD-Scenes: A Next-Generation Multimodal Dataset for Autonomous Driving

MIPD: A Multi-sensory Interactive Perception Dataset for Embodied Intelligent Driving

ZUST Campus: A Lightweight and Practical LiDAR SLAM Dataset for Autonomous Driving Scenarios

A Benchmark for Multi-Modal LiDAR SLAM with Ground Truth in GNSS-Denied Environments

NTU4DRadLM: 4D Radar-centric Multi-Modal Dataset for Localization and Mapping

Ground-Challenge: A Multi-sensor SLAM Dataset Focusing on Corner Cases for Ground Robots

IAMCV Multi-Scenario Vehicle Interaction Dataset

End-to-End Multimodal Sensor Dataset Collection Framework for Autonomous Vehicles

TAIL: A Terrain-Aware Multi-Modal SLAM Dataset for Robot Locomotion in Deformable Granular Environments

The ADUULM-360 Dataset -- A Multi-Modal Dataset for Depth Estimation in Adverse Weather

EnvoDat: A Large-Scale Multisensory Dataset for Robotic Spatial Awareness and Semantic Reasoning in Heterogeneous Environments