A9 Intersection Dataset: All You Need for Urban 3D Camera-LiDAR Roadside Perception

Walter Zimmer,Christian Creß,Huu Tung Nguyen,Alois C. Knoll
2023-06-16
Abstract:Intelligent Transportation Systems (ITS) allow a drastic expansion of the visibility range and decrease occlusions for autonomous driving. To obtain accurate detections, detailed labeled sensor data for training is required. Unfortunately, high-quality 3D labels of LiDAR point clouds from the infrastructure perspective of an intersection are still rare. Therefore, we provide the A9 Intersection Dataset, which consists of labeled LiDAR point clouds and synchronized camera images. Here, we recorded the sensor output from two roadside cameras and LiDARs mounted on intersection gantry bridges. The point clouds were labeled in 3D by experienced annotators. Furthermore, we provide calibration data between all sensors, which allow the projection of the 3D labels into the camera images and an accurate data fusion. Our dataset consists of 4.8k images and point clouds with more than 57.4k manually labeled 3D boxes. With ten object classes, it has a high diversity of road users in complex driving maneuvers, such as left and right turns, overtaking, and U-turns. In experiments, we provided multiple baselines for the perception tasks. Overall, our dataset is a valuable contribution to the scientific community to perform complex 3D camera-LiDAR roadside perception tasks. Find data, code, and more information at <a class="link-external link-https" href="https://a9-dataset.com" rel="external noopener nofollow">this https URL</a>.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
### What problem does this paper attempt to solve? This paper aims to address the issue of 3D perception from a roadside perspective in Intelligent Transportation Systems (ITS). Specifically: 1. **Lack of datasets**: Existing datasets are primarily collected from vehicle perspectives or aerial views, lacking high-quality roadside perspective LiDAR point cloud datasets. 2. **Insufficient label quality**: Some existing datasets have deficiencies in label quality (e.g., occluded areas filled with rectangles), lack certain vehicle categories (e.g., trucks and buses), or have insufficient data volume. ### The main contributions of the paper include: 1. **A9 Intersection Dataset**: Provides a dataset containing 4800 annotated LiDAR point cloud frames and synchronized camera images, covering complex driving scenarios (such as left turns, right turns, overtaking, etc.) with ten different road participant categories. 2. **High-precision annotation**: All point cloud frames are manually annotated by experts and provide detailed attribute information. 3. **Sensor calibration data**: Offers extrinsic calibration data between LiDAR and cameras, supporting early and late fusion. 4. **A9-Devkit**: Provides data loaders, converters, and visualization modules, facilitating researchers to use this dataset for experiments. Through these contributions, the paper aims to provide a high-quality dataset for 3D perception tasks in intelligent transportation systems, thereby advancing research in the related field.