Abstract:Background: Endoscopic instrument segmentation is essential for ensuring the safety of robotic-assisted spinal endoscopic surgeries. However, due to the narrow operative region, intricate surrounding tissues, and limited visibility, achieving instrument segmentation within the endoscopic view remains challenging. Purpose: This work aims to devise a method to segment surgical instruments in endoscopic video. By designing an endoscopic image classification model, features of frames before and after the video are extracted to achieve continuous and precise segmentation of instruments in endoscopic videos. Methods: Deep learning techniques serve as the algorithmic core for constructing the convolutional neural network proposed in this study. The method comprises dual stages: image classification and instrument segmentation. MobileViT is employed for image classification, enabling the extraction of key features of different instruments and generating classification results. DeepLabv3+ is utilized for instrument segmentation. By training on distinct instruments separately, corresponding model parameters are obtained. Lastly, a flag caching mechanism along with a blur detection module is designed to effectively utilize the image features in consecutive frames. By incorporating specific parameters into the segmentation model, better segmentation of surgical instruments can be achieved in endoscopic videos. Results: The classification and segmentation models are evaluated on an endoscopic image dataset. In the dataset used for instrument segmentation, the training set consists of 7456 images, the validation set consists of 829 images, and the test set consists of 921 images. In the dataset used for image classification, the training set consists of 2400 images and the validation set consists of 600 images. The image classification model achieves an accuracy of 70% on the validation set. For the segmentation model, experiments are conducted on two common surgical instruments, and the mean Intersection over Union (mIoU) exceeds 98%. Furthermore, the proposed video segmentation method is tested using videos collected during surgeries, validating the effectiveness of the flag caching mechanism and blur detection module. Conclusions: Experimental results on the dataset demonstrate that the dual-stage video processing method excels in performing instrument segmentation tasks under endoscopic conditions. This advancement is significant for enhancing the intelligence level of robotic-assisted spinal endoscopic surgeries.

SSIS-Seg: Simulation-Supervised Image Synthesis for Surgical Instrument Segmentation

Lactation and reproduction.

Synthetic and Real Inputs for Tool Segmentation in Robotic Surgery

Data Augmentation for Surgical Scene Segmentation with Anatomy-Aware Diffusion Models

Generalizing Surgical Instruments Segmentation to Unseen Domains with One-to-Many Synthesis

Real-time instance segmentation of surgical instruments using attention and multi-scale feature fusion

SimuScope: Realistic Endoscopic Synthetic Dataset Generation through Surgical Simulation and Diffusion Models

Searching for Efficient Architecture for Instrument Segmentation in Robotic Surgery

Local Style Preservation in Improved GAN-Driven Synthetic Image Generation for Endoscopic Tool Segmentation

Image Synthesis with Class-Aware Semantic Diffusion Models for Surgical Scene Segmentation

Realistic Surgical Image Dataset Generation Based On 3D Gaussian Splatting

Can surgical simulation be used to train detection and classification of neural networks?

Deep Residual Learning for Instrument Segmentation in Robotic Surgery

Real-Time Instrument Segmentation in Robotic Surgery using Auxiliary Supervised Deep Adversarial Learning

Dual-stage semantic segmentation of endoscopic surgical instruments

Revisiting Surgical Instrument Segmentation Without Human Intervention: A Graph Partitioning View

IMAGINING WHAT YOU DON ' T KNOW : THE THEORETICAL GOALS OF THE ROSSETTI ARCHIVE

Handling Geometric Domain Shifts in Semantic Segmentation of Surgical RGB and Hyperspectral Images

Privacy-Preserving Synthetic Continual Semantic Segmentation for Robotic Surgery

Navigating the Synthetic Realm: Harnessing Diffusion-based Models for Laparoscopic Text-to-Image Generation

Towards Unsupervised Learning for Instrument Segmentation in Robotic Surgery with Cycle-Consistent Adversarial Networks