Fast Connected Components Object Segmentation on Fused Lidar and Stereo-Camera Point Clouds with Visual-Inertial-Gimbal for Mobile Applications Utilizing GPU Acceleration

Martin Hünermund,Maik Groneberg,Artur Schütz

DOI: https://doi.org/10.1007/978-3-030-68476-1_7

2021-01-01

Abstract:Computer vision systems in mobile applications require low-latency responses and a high framerate due to the highly dynamic environment. Autonomous driving computer vision systems are often based on regular cameras, stereo cameras or on Lidar technology. Recently developed AI methods are often used for object detection in image data. These learning-based methods require a large amount of training data, the quality of its outputs highly depend on the quality of this training data and they rarely achieve high frame rates. Lidar has a high reputation for safety applications. Yet it suffers from a high price and a low number of points to reason from. Image-based computer vision can derive information from a much greater amount of data. Stereo-cameras derive depth information from two images by the optical principle of parallax. This can be used to enrich the already safe lidar, to make a more reliable sensor system.Sensors in mobile applications are subject to orientation changes, movement and vibrations. These negatively influence the data sampling process and therefore need to be compensated for. Lidar and stereo cameras have other sensor effects like flying points, which need to be filtered.We propose a high-performance object segmentation system on Lidar point clouds fused with stereo camera point clouds. For short term motion and vibration compensation, we propose the use of a visual-inertial-sensor based virtual gimbal. Modern graphics processing units (GPU) are used for sensor filtering, point cloud fusion, voxelization and connected component segmentation to achieve the high framerate and low-latency necessary for use in a dynamic environment.The object segmentation system is used for the collision avoidance system of an autonomous cargo bike developed in the research project RavE-Bike.