BAMF-SLAM: Bundle Adjusted Multi-Fisheye Visual-Inertial SLAM Using Recurrent Field Transforms

Wei Zhang,Sen Wang,Xingliang Dong,Rongwei Guo,Norbert Haala
2023-06-15
Abstract:In this paper, we present BAMF-SLAM, a novel multi-fisheye visual-inertial SLAM system that utilizes Bundle Adjustment (BA) and recurrent field transforms (RFT) to achieve accurate and robust state estimation in challenging scenarios. First, our system directly operates on raw fisheye images, enabling us to fully exploit the wide Field-of-View (FoV) of fisheye cameras. Second, to overcome the low-texture challenge, we explore the tightly-coupled integration of multi-camera inputs and complementary inertial measurements via a unified factor graph and jointly optimize the poses and dense depth maps. Third, for global consistency, the wide FoV of the fisheye camera allows the system to find more potential loop closures, and powered by the broad convergence basin of RFT, our system can perform very wide baseline loop closing with little overlap. Furthermore, we introduce a semi-pose-graph BA method to avoid the expensive full global BA. By combining relative pose factors with loop closure factors, the global states can be adjusted efficiently with modest memory footprint while maintaining high accuracy. Evaluations on TUM-VI, Hilti-Oxford and Newer College datasets show the superior performance of the proposed system over prior works. In the Hilti SLAM Challenge 2022, our VIO version achieves second place. In a subsequent submission, our complete system, including the global BA backend, outperforms the winning approach.
Robotics
What problem does this paper attempt to address?
This paper aims to solve the problem of state estimation with high precision and robustness in challenging environments. Specifically, the paper proposes a multi - fisheye visual - inertial SLAM system named BAMF - SLAM. This system utilizes Bundle Adjustment (BA) and Recurrent Field Transforms (RFT) techniques to achieve accurate state estimation in complex scenarios such as low - texture, rapid illumination changes and motion. ### Main problems and solutions 1. **Tracking problems in low - texture environments**: - The paper uses multiple fisheye cameras to increase the Field - of - View (FoV), thereby reducing the influence of low - texture areas and increasing the field - of - view overlap rate. - The system also combines multi - camera inputs and complementary inertial measurement data and jointly optimizes poses and dense depth maps through a unified factor graph to overcome the low - texture challenge. 2. **Global consistency problems**: - The wide Field - of - View (FoV) of fisheye cameras enables the system to find more potential closed - loops, thereby improving global consistency. - The system introduces a semi - pose - graph Bundle Adjustment (BA) method. By combining relative pose factors with closed - loop factors, it efficiently adjusts the global state while maintaining high precision. 3. **Computational efficiency problems**: - To reduce computational overhead, the paper proposes a semi - pose - graph Bundle Adjustment method, avoiding the expensive computational cost of full - global Bundle Adjustment. - By converting the front - end reprojection factors into relative pose factors and constructing a pose graph, the system can significantly improve computational efficiency while maintaining high precision. ### Technical details - **Fisheye camera model**: Use the Kannala - Brandt fisheye camera model to directly process raw fisheye images and fully utilize the advantages of its wide Field - of - View. - **IMU initialization**: Initialize the IMU state through a series of steps, including visual - only BA, gravity direction estimation, inertial - only BA and visual - inertial BA, to ensure that the system can accurately fuse visual and inertial information. - **Factor graph representation**: The system adopts factor graph representation, which contains multiple types of factors, such as monocular reprojection factors, stereo reprojection factors and IMU pre - integration factors, to jointly optimize multi - camera inputs and IMU measurement data. - **Semi - pose - graph Bundle Adjustment**: By converting the front - end reprojection factors into relative pose factors, construct a pose graph, and combine closed - loop factors for global optimization to improve the global consistency and accuracy of the system. ### Experimental results - **TUM - VI dataset**: In multiple challenging indoor scenarios, the BAMF - SLAM system outperforms other methods in terms of the Mean Absolute Trajectory Error (ATE). - **Newer College dataset**: In datasets containing indoor and large - scale outdoor scenarios, the BAMF - SLAM system performs excellently in terms of the Relative Pose Error (RPE) and ATE. - **Hilti - Oxford dataset**: In the Hilti SLAM Challenge 2022, the BAMF - SLAM system achieved results superior to other methods on multiple sequences, especially in scenarios requiring closed - loops. In conclusion, through proposing the BAMF - SLAM system, this paper effectively solves the problem of achieving high - precision and robust state estimation in complex environments, providing new ideas and technical support for the development of multi - fisheye visual - inertial SLAM systems.