UniDet3D: Multi-dataset Indoor 3D Object Detection

Maksim Kolodiazhnyi,Anna Vorontsova,Matvey Skripkin,Danila Rukhovich,Anton Konushin

2024-09-06

Abstract:Growing customer demand for smart solutions in robotics and augmented reality has attracted considerable attention to 3D object detection from point clouds. Yet, existing indoor datasets taken individually are too small and insufficiently diverse to train a powerful and general 3D object detection model. In the meantime, more general approaches utilizing foundation models are still inferior in quality to those based on supervised training for a specific task. In this work, we propose \ours{}, a simple yet effective 3D object detection model, which is trained on a mixture of indoor datasets and is capable of working in various indoor environments. By unifying different label spaces, \ours{} enables learning a strong representation across multiple datasets through a supervised joint training scheme. The proposed network architecture is built upon a vanilla transformer encoder, making it easy to run, customize and extend the prediction pipeline for practical use. Extensive experiments demonstrate that \ours{} obtains significant gains over existing 3D object detection methods in 6 indoor benchmarks: ScanNet (+1.1 mAP50), ARKitScenes (+19.4 mAP25), S3DIS (+9.1 mAP50), MultiScan (+9.3 mAP50), 3RScan (+3.2 mAP50), and ScanNet++ (+2.7 mAP50). Code is available at <a class="link-external link-https" href="https://github.com/filapro/unidet3d" rel="external noopener nofollow">this https URL</a> .

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that when existing indoor 3D object detection methods are trained on a single dataset, due to insufficient data volume and lack of diversity, the generalization ability of the model is poor and it cannot perform well in different indoor environments. Specifically:

1. **Insufficient dataset size and diversity**: When used alone, existing indoor 3D datasets are small in size and not diverse enough to train a powerful and general - purpose 3D object detection model.
2. **Poor cross - dataset generalization ability**: Although some supervised - learning - based methods perform well on specific tasks, their generalization ability between different datasets is poor.
3. **Limitations of existing methods**: Existing multi - dataset training methods either use different architectures and weights or have poor performance during joint training and cannot fully utilize the advantages of multiple datasets.

To address these problems, the paper proposes **UniDet3D**, which is a simple and effective 3D object detection model that can work in various indoor environments by training with a mixture of multiple indoor datasets. Specifically, UniDet3D solves the above problems in the following ways:

- **Unified label space**: By merging the label spaces of different datasets, the model can learn powerful representations across multiple datasets.
- **Simple network architecture**: Based on a pure self - attention encoder architecture, without position encoding and cross - attention, making the model easy to run, customize, and expand.
- **Joint training scheme**: Through a supervised joint training scheme, the model can perform excellently in multiple indoor environments.

Experimental results show that UniDet3D has achieved significant performance improvements on six indoor benchmark datasets (ScanNet, ARKitScenes, S3DIS, MultiScan, 3RScan, and ScanNet++), especially on the S3DIS dataset, where the mAP 50 has increased by 9.1%. These results prove the effectiveness of UniDet3D in improving the generalization ability of the model and detection accuracy.