3D car-detection based on a Mobile Deep Sensor Fusion Model and real-scene applications

Qiang Zhang,Xiaojian Hu,Ziyi Su,Zhihong Song
DOI: https://doi.org/10.1371/journal.pone.0236947
IF: 3.7
2020-09-03
PLoS ONE
Abstract:Unmanned vehicles need to make a comprehensive perception of the surrounding environmental information during driving. Perception of automotive information is of significance. In the field of automotive perception, the sterevision of car-detection plays a vital role and sterevision can calculate the length, width, and height of a car, making the car more specific. However, under the existing technology, it is impossible to obtain accurate detection in a complex environment by relying on a single sensor. Therefore, it is particularly important to study the complex sensing technology based on multi-sensor fusion. Recently, with the development of deep learning in the field of vision, a mobile sensor-fusion method based on deep learning is proposed and applied in this paper--Mobile Deep Sensor Fusion Model (MDSFM). The content of this article is as follows. It does a data processing that projects 3D data to 2D data, which can form a dataset suitable for the model, thereby training data more efficiently. In the modules of LiDAR, it uses a revised squeezeNet structure to lighten the model and reduce parameters. In the modules of cameras, it uses the improved design of detecting module in R-CNN with a Mobile Spatial Attention Module (MSAM). In the fused part, it uses a dual-view deep fusing structure. And then it selects images from the KITTI's datasets for validation to test this model. Compared with other recognized methods, it shows that our model has a fairly good performance. Finally, it implements a ROS program on the experimental car and our model is in good condition. The result shows that it can improve performance of detecting easy cars significantly through MDSFM. It increases the quality of the detected data and improves the generalized ability of car-detection model. It improves contextual relevance and preserves background information. It remains stable in driverless environments. It is applied in the realistic scenario and proves that the model has a good practical value.
multidisciplinary sciences
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to achieve high - precision 3D vehicle detection in complex environments. Specifically, the paper focuses on the comprehensive perception of the surrounding environment information by unmanned vehicles during driving, especially the perception of vehicle information. Existing technologies cannot obtain accurate detection results in complex environments when relying on a single sensor. Therefore, it has become particularly important to study complex perception technologies based on multi - sensor fusion. To meet this challenge, the paper proposes a mobile sensor fusion method based on deep learning - the Mobile Deep Sensor Fusion Model (MDSFM). This model improves the performance of 3D vehicle detection in the following ways: 1. **Data Processing**: Project 3D data onto 2D data to form a data set suitable for the model, thereby training data more efficiently. 2. **LiDAR Module**: Use an improved SqueezeNet structure to reduce the weight of the model and decrease the number of parameters. 3. **Camera Module**: Use an improved R - CNN detection module and the Mobile Spatial Attention Module (MSAM). 4. **Fusion Part**: Adopt a two - view depth fusion structure. 5. **Verification**: Select images from the KITTI data set for verification to test the performance of the model. 6. **Practical Application**: Implement the ROS program on the experimental vehicle and prove the practical value of this model in real - life scenarios. Through these methods, MDSFM significantly improves the performance of simple vehicle detection, enhances the quality of detection data, strengthens the generalization ability of the model, improves context - relatedness, and retains background information. In addition, this model remains stable in an unmanned driving environment and has good practical application value.