Accurate and Interactive Visual-Inertial Sensor Calibration with Next-Best-View and Next-Best-Trajectory Suggestion

Christopher L. Choi,Binbin Xu,Stefan Leutenegger
DOI: https://doi.org/10.48550/arXiv.2309.14514
2023-09-26
Abstract:Visual-Inertial (VI) sensors are popular in robotics, self-driving vehicles, and augmented and virtual reality applications. In order to use them for any computer vision or state-estimation task, a good calibration is essential. However, collecting informative calibration data in order to render the calibration parameters observable is not trivial for a non-expert. In this work, we introduce a novel VI calibration pipeline that guides a non-expert with the use of a graphical user interface and information theory in collecting informative calibration data with Next-Best-View and Next-Best-Trajectory suggestions to calibrate the intrinsics, extrinsics, and temporal misalignment of a VI sensor. We show through experiments that our method is faster, more accurate, and more consistent than state-of-the-art alternatives. Specifically, we show how calibrations with our proposed method achieve higher accuracy estimation results when used by state-of-the-art VI Odometry as well as VI-SLAM approaches. The source code of our software can be found on: <a class="link-external link-https" href="https://github.com/chutsu/yac" rel="external noopener nofollow">this https URL</a>.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
### Problems Addressed by the Paper This paper aims to address the calibration problem of Visual-Inertial (VI) sensors. Specifically, it proposes a new interactive VI sensor calibration process that guides non-expert users to collect effective calibration data through a graphical user interface and information theory, thereby improving the accuracy, consistency, and efficiency of calibration. ### Background and Motivation 1. **Limitations of Traditional Methods**: - Traditional VI sensor calibration usually requires expert operation, collecting calibration data by placing the sensor in front of a calibration board and moving it. - Non-expert users often do not know how to choose appropriate calibration views and motion ranges, leading to insufficient or excessive data collection. - Existing calibration tools like Kalibr, although effective, rely on high-quality calibration data and require multiple attempts to achieve satisfactory results. 2. **Limitations of Online Methods**: - Online state estimation frameworks like OKVIS, VINS-MONO, and OpenVINS can estimate calibration parameters in real-time but require initial accurate calibration and sufficient visual features and motion excitation. - These methods still face similar issues as offline calibration, and natural keypoints and lack of precise 3D position knowledge may not produce optimal results. ### Proposed Method 1. **Interactive Calibration Process**: - Guides users to collect effective calibration data through a graphical user interface and information theory. - Uses Next-Best-View (NBV) and Next-Best-Trajectory (NBT) suggestions to help users determine the best next view and trajectory. - This method is suitable for any number of cameras and supports real-time feedback. 2. **Specific Steps**: - **First Stage**: Calibrate camera intrinsic and extrinsic parameters using only visual data. - **Second Stage**: Based on fixed camera intrinsic and extrinsic parameters, calibrate camera-IMU extrinsic parameters and time delay through NBT suggestions. ### Experimental Validation 1. **Offline Experiments**: - The method was validated using the EuRoC dataset in offline mode, showing competitive calibration accuracy compared to Kalibr. - Results indicate that the method's RMSE reprojection error in both camera calibration and camera-IMU calibration stages is better than or close to Kalibr. 2. **User Experiments**: - Experiments involving 16 graduate students validated the method's effectiveness in practical applications. - Results show that even beginners can achieve better calibration results than Kalibr using this method, especially in VIO mode. ### Main Contributions 1. **A complete open-source interactive VI camera calibration tool** that supports any number of cameras. 2. **An information-theory-based method** for identifying the most effective NBV and NBT. 3. **An interactive graphical user interface** that guides users through the calibration data collection process. 4. **Experimental validation** demonstrating that the method outperforms existing non-interactive calibration methods in terms of speed, accuracy, and reliability, even when used by beginners. ### Conclusion This paper proposes an interactive VI sensor calibration method that addresses the difficulties non-expert users face in collecting effective calibration data, significantly improving calibration accuracy and consistency. The method has been validated in multiple experiments, showing superior performance in practical applications.