Abstract:In numerous robotic and autonomous driving tasks, traditional visual SLAM algorithms estimate the camera's position in a scene through sparse feature points and express the map by estimating the depth of sparse point clouds. However, practical applications require SLAM to create dense maps in real time, overcoming the sparsity and occlusion issues of point clouds. Furthermore, it is advantageous for SLAM map to possess an auto-completion capability, where the map can automatically infer and complete the remaining 20% when the camera observes only 80% of an object. Therefore, a more dense and intelligent map representation is needed. In this paper, we propose a Visual–Inertial SLAM with Neural Radiance Fields reconstruction to address the aforementioned challenges. We integrate the traditional rule-based optimization with NeRF. This approach allows for the real-time update of NeRF local functions by rapidly estimating camera motion and sparse feature point depths to reconstruct 3D scenes. To achieve better camera poses and globally consistent map, we address the issue of IMU noise spikes resulting from rapid motion changes, along with handling pose adjustments due to loop closure fusion. Specifically, we employ a form of widening the static noise covariance to refit the dynamic noise covariance. During loop closure fusion, we treat the pose adjustment between pre- and post-loop closure as a spatiotemporal transformation, migrating NeRF parameters from pre- to post- to expedite loop closure adjustments in NeRF mapping. Moreover, we extend this method to scenarios with only grayscale images. By expanding the color channels of grayscale images and conducting linear spatial mapping, we can rapidly reconstruct 3D scenes with only grayscale images. We demonstrate the precision and speed advantages of our method in both RGB and grayscale scenes.

A Study of Structured Light Based 3-D Vision Inspection Using RBF Neural Network

Three dimensional vision inspection based on structured light projection and neurocalibration

Objective Quality Assessment of Retargeted Images Based on RBF Neural Network with Structural Distortion and Content Change

A Calibration Method for Structured Light 3-D Vision Systems

A novel calibration approach to structured light 3D vision inspection

Study on Illumination Measurement Method of Lighting Environment Based on RBF Neural Network

A Novel Camera Calibration Method for Binocular Vision Based on Improved RBF Neural Network

A High Precision Approach to Calibrate a Structured Light Vision Sensor in a Robot-Based Three-Dimensional Measurement System

Linear CCD camera calibration of exterior attitude measurement system based on BP neural network

NeRF-Loc: Visual Localization with Conditional Neural Radiance Field.

Fast Parallel 3D Stereo Vision Measurement System Using an Artificial Neural Network

Complex object 3D measurement based on phase-shifting and a neural network

A three-dimensional vision measurement method based on double-line combined structured light

Application of RBF Neural Network in Three Dimensional Laser Cutting

Research on Structured Light 3D Vision in the Remanufacturing System Based on Robotic Arc Welding

VI-NeRF-SLAM: a real-time visual–inertial SLAM with NeRF mapping

Structured Light Based High Precision 3D Measurement and Workpiece Pose Estimation

Irregularity Inspection using Neural Radiance Field

Depth Reconstruction with Neural Signed Distance Fields in Structured Light Systems

Structured-light 3D Shape Measurements Using Deep Learning

The hybrid calibration of linear structured light system.