ViLiVO: Virtual LiDAR-Visual Odometry for an Autonomous Vehicle with a Multi-Camera System

Zhenzhen Xiang,Jingrui Yu,Jie Li,Jianbo Su

DOI: https://doi.org/10.48550/arXiv.1909.12947

2019-09-30

Abstract:In this paper, we present a multi-camera visual odometry (VO) system for an autonomous vehicle. Our system mainly consists of a virtual LiDAR and a pose tracker. We use a perspective transformation method to synthesize a surround-view image from undistorted fisheye camera images. With a semantic segmentation model, the free space can be extracted. The scans of the virtual LiDAR are generated by discretizing the contours of the free space. As for the pose tracker, we propose a visual odometry system fusing both the feature matching and the virtual LiDAR scan matching results. Only those feature points located in the free space area are utilized to ensure the 2D-2D matching for pose estimation. Furthermore, bundle adjustment (BA) is performed to minimize the feature points reprojection error and scan matching error. We apply our system to an autonomous vehicle equipped with four fisheye cameras. The testing scenarios include an outdoor parking lot as well as an indoor garage. Experimental results demonstrate that our system achieves a more robust and accurate performance comparing with a fisheye camera based monocular visual odometry system.

Computer Vision and Pattern Recognition,Robotics

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to achieve robust and accurate visual odometry (VO) using a multi - eye fisheye camera system on autonomous vehicles. Specifically, the authors propose a system named ViLiVO (Virtual LiDAR - Visual Odometry). This system improves the accuracy and robustness of vehicle positioning by combining virtual LiDAR and feature matching. Traditional monocular or binocular visual odometry may fail in the face of rapid motion, environmental changes or lack of texture, while ViLiVO provides more stable performance in various driving scenarios by introducing virtual LiDAR scan data and combining texture information and geometric information.

The main contributions of the paper are as follows:
1. **Design of virtual LiDAR**: Extract free - space areas from multi - eye fisheye camera images through semantic segmentation technology and generate virtual LiDAR scan points, which are used for pose estimation.
2. **Multi - sensor fusion**: Fuse the scan - matching results of virtual LiDAR with the feature - point - based matching results. Through the bundle adjustment (BA) optimization algorithm, the accuracy of pose estimation is improved.
3. **Robustness improvement**: By combining visual features and geometric information, the system performs more stably in dynamic environments, especially in cases of rapid motion or large environmental changes.

The performance of the ViLiVO system is verified by experiments in the paper. The results show that in indoor and outdoor scenarios, ViLiVO has a lower drift rate and higher accuracy compared with traditional monocular visual odometry systems.