Monocular Semantic SLAM using Object-pose-graph Constraints

Xiu Li,Yuwang Wang,Yuhan Liu,Qionghai Dai

2018-01-01

Abstract:We present a novel approach for monocular semantic SLAM, which is enhanced by category-specific object-level constraints to achieve more robust in ego-motion estimation, trajectory optimization and environment mapping. We extract the semantic information from input images by utilizing stateof-the-art convolutional neural networks(CNNs) for 2D object detection and semantic keypoints localization. In contrast with conventional SLAM system which takes the environment as point clouds, both the objects and semantic keypoints provide internal correspondences among image sequences and act as salient parts for SLAM system. The object-pose-graph is built based on conventional keyframe-pose-graph by containing constraints between the object 3D pose and camera pose. Then the 3D pose of the objects and camera trajectory are collaboratively optimized. The 3D objects estimated in object-pose-graph provide reliable camera tracking under pure rotational camera motion. By taking the objects as loop detection cues, we are able to close the loop when the object is visible even in significant variant viewpoint which outperforms traditional loop closure detection. We estimated our object-pose-graph semantic SLAM system on both indoor and outdoor environment and achieve better camera tracking and mapping result.