Abstract:Fetoscopic laser ablation surgery could be guided using a high-quality panorama of the operating site, representing a map of the placental vasculature. This can be achieved during the initial inspection phase of the procedure using image mosaicking techniques. Due to the lack of camera calibration in the operating room, it has been mostly modelled as an affine registration problem. While previous work mostly focuses on image feature extraction for visual odometry, the challenges related to large-scale reconstruction (re-localisation, loop closure, drift correction) remain largely unaddressed in this context. This letter proposes using pose graph optimisation to produce globally optimal image mosaics of placental vessels. Our approach follows the SLAM framework with a front-end for visual odometry and a back-end for long-term refinement. Our front-end uses a recent state-of-the-art odometry approach based on vessel segmentation, which is then managed by a key-frame structure and the bag-of-words (BoW) scheme to retrieve loop closures. The back-end, which is our key contribution, models odometry and loop closure constraints as a pose graph with affine warpings between states. This problem in the special Euclidean space cannot be solved by existing pose graph algorithms and available libraries such as G2O. We model states on affine Lie group with local linearisation in its Lie algebra. The cost function is established using Mahalanobis distance with the vectorisation of the Lie algebra. Finally, an iterative optimisation algorithm is adopted to minimise the cost function. The proposed pose graph optimisation is first validated on simulation data with a synthetic trajectory that has different levels of noise and different numbers of loop closures. Then the whole system is validated using real fetoscopic data that has three sequences with different numbers of frames and loop closures. Experimental results validate the advantage of the proposed method compared with baselines.

Globally Optimal Fetoscopic Mosaicking Based on Pose Graph Optimisation with Affine Constraints

Retrieval and registration of long-range overlapping frames for scalable mosaicking of in vivo fetoscopy

Deep Sequential Mosaicking of Fetoscopic Videos

Robust Endoscopic Image Mosaicking Via Fusion of Multimodal Estimation

Deep Placental Vessel Segmentation for Fetoscopic Mosaicking

Local Optimized and Scalable Frame-to-model SLAM

Ct-LVI: A Framework Toward Continuous-Time Laser-Visual-Inertial Odometry and Mapping

A SLAM Pose Graph Optimization Method Using Dual Visual Odometry

FetusMapV2: Enhanced Fetal Pose Estimation in 3D Ultrasound

SCE-LIO: An Enhanced LiDAR Inertial Odometry by Constructing Submap Constraints

Fast Georeferenced Aerial Image Stitching With Absolute Rotation Averaging and Planar- Restricted Pose Graph

Learning-based keypoint registration for fetoscopic mosaicking

RTSfM: Real-Time Structure From Motion for Mosaicing and DSM Mapping of Sequential Aerial Images With Low Overlap

Optimized SC-F-LOAM: Optimized Fast LiDAR Odometry and Mapping Using Scan Context

MambaVO: Deep Visual Odometry Based on Sequential Matching Refinement and Training Smoothing

An Efficient Visual SfM Framework Using Planar Markers.

A Global Approach for Automatic Fibroscopic Video Mosaicing in Minimally Invasive Diagnosis

FetReg: Placental Vessel Segmentation and Registration in Fetoscopy Challenge Dataset

Learning How To Robustly Estimate Camera Pose in Endoscopic Videos

Non-convex Pose Graph Optimization in SLAM via Proximal Linearized Riemannian ADMM