Dynamic Reconstruction of Deformable Soft-tissue with Stereo Scope in Minimal Invasive Surgery

Jingwei Song,Jun Wang,Liang Zhao,Shoudong Huang,Gamini Dissanayake

DOI: https://doi.org/10.1109/LRA.2017.2735487

2020-03-23

Abstract:In minimal invasive surgery, it is important to rebuild and visualize the latest deformed shape of soft-tissue surfaces to mitigate tissue damages. This paper proposes an innovative Simultaneous Localization and Mapping (SLAM) algorithm for deformable dense reconstruction of surfaces using a sequence of images from a stereoscope. We introduce a warping field based on the Embedded Deformation (ED) nodes with 3D shapes recovered from consecutive pairs of stereo images. The warping field is estimated by deforming the last updated model to the current live model. Our SLAM system can: (1) Incrementally build a live model by progressively fusing new observations with vivid accurate texture. (2) Estimate the deformed shape of unobserved region with the principle As-Rigid-As-Possible. (3) Show the consecutive shape of models. (4) Estimate the current relative pose between the soft-tissue and the scope. In-vivo experiments with publicly available datasets demonstrate that the 3D models can be incrementally built for different soft-tissues with different deformations from sequences of stereo images obtained by laparoscopes. Results show the potential clinical application of our SLAM system for providing surgeon useful shape and texture information in minimal invasive surgery.

Computer Vision and Pattern Recognition,Robotics,Image and Video Processing

What problem does this paper attempt to address?

This paper attempts to address the problem of real-time reconstruction and visualization of soft tissue deformation shapes in Minimally Invasive Surgery (MIS). Specifically, the paper proposes an innovative Simultaneous Localization and Mapping (SLAM) algorithm that utilizes image sequences obtained from a stereo endoscope to achieve dense deformation reconstruction of the soft tissue surface.

### Main Issues
1. **Limited Field of View**: The field of view of the endoscope is very narrow, resulting in limited information observed in each frame.
2. **Smooth Surfaces**: Most soft tissue surfaces are smooth and lack distinct geometric features, posing challenges for the registration process.
3. **Rapid Motion**: Due to the rapid movement of the camera, breathing, heartbeat, and instrument interaction, both rigid and non-rigid motions of the soft tissue increase the difficulty of reconstruction and visualization.

### Solutions
1. **Dense Point Cloud Management**: The paper adopts a point cloud data management method instead of the traditional volumetric model management method, avoiding the problem of blurred surfaces and allowing simultaneous management of texture/color information.
2. **Dense SURF Feature Point Extraction**: To overcome the drift problem caused by smooth surfaces, the paper uses a dense SURF algorithm to extract a large number of feature points, improving the stability of registration.
3. **Embedded Deformation (ED) Nodes**: By using embedded deformation nodes and affine matrices to estimate the deformation field, the non-rigid deformation modeling of soft tissue is achieved.
4. **Energy Function Optimization**: By minimizing an energy function that includes rotation constraints, regularization terms, point-to-plane distances, and sparse feature point correspondences, the deformation parameters are optimized.
5. **Real-time Fusion of New Observations**: By using a weighted point cloud method, the current depth is fused with the new depth generated by the model to update the model.

### Experimental Validation
The paper uses an in vivo stereo video dataset provided by the Hamlyn Centre for validation. Experimental results show that the SLAM system can incrementally build 3D models on different types of soft tissues and has potential clinical application value, providing useful shape and texture information for surgeons.

### Conclusion
The paper proposes an innovative SLAM framework that can achieve real-time reconstruction and visualization of soft tissue deformation shapes in minimally invasive surgery. Through techniques such as point cloud data management and dense SURF feature point extraction, it effectively addresses the limitations of traditional methods in minimally invasive surgery.