Abstract:Background: Endoscopic instrument segmentation is essential for ensuring the safety of robotic-assisted spinal endoscopic surgeries. However, due to the narrow operative region, intricate surrounding tissues, and limited visibility, achieving instrument segmentation within the endoscopic view remains challenging. Purpose: This work aims to devise a method to segment surgical instruments in endoscopic video. By designing an endoscopic image classification model, features of frames before and after the video are extracted to achieve continuous and precise segmentation of instruments in endoscopic videos. Methods: Deep learning techniques serve as the algorithmic core for constructing the convolutional neural network proposed in this study. The method comprises dual stages: image classification and instrument segmentation. MobileViT is employed for image classification, enabling the extraction of key features of different instruments and generating classification results. DeepLabv3+ is utilized for instrument segmentation. By training on distinct instruments separately, corresponding model parameters are obtained. Lastly, a flag caching mechanism along with a blur detection module is designed to effectively utilize the image features in consecutive frames. By incorporating specific parameters into the segmentation model, better segmentation of surgical instruments can be achieved in endoscopic videos. Results: The classification and segmentation models are evaluated on an endoscopic image dataset. In the dataset used for instrument segmentation, the training set consists of 7456 images, the validation set consists of 829 images, and the test set consists of 921 images. In the dataset used for image classification, the training set consists of 2400 images and the validation set consists of 600 images. The image classification model achieves an accuracy of 70% on the validation set. For the segmentation model, experiments are conducted on two common surgical instruments, and the mean Intersection over Union (mIoU) exceeds 98%. Furthermore, the proposed video segmentation method is tested using videos collected during surgeries, validating the effectiveness of the flag caching mechanism and blur detection module. Conclusions: Experimental results on the dataset demonstrate that the dual-stage video processing method excels in performing instrument segmentation tasks under endoscopic conditions. This advancement is significant for enhancing the intelligence level of robotic-assisted spinal endoscopic surgeries.

Automated detection of steps in videos of strabismus surgery using deep learning

Intelligent cataract surgery supervision and evaluation via deep learning

Automated Strabismus Detection for Telemedicine Applications

A Deep Learning-Based Image Analysis for Assessing the Extent of Abduction in Abducens Nerve Palsy Patients Before and after Strabismus Surgery

A Fully Automatic Postoperative Appearance Prediction System for Blepharoptosis Surgery with Image-based Deep Learning

Automated Photographic Analysis of Inferior Oblique Overaction Based on Deep Learning.

Efficient Surgical Tool Recognition via HMM-Stabilized Deep Learning

Vitreoretinal Surgical Instrument Tracking in Three Dimensions Using Deep Learning

Beyond PhacoTrainer: Deep Learning for Enhanced Trabecular Meshwork Detection in MIGS Videos

Surgical Phase Recognition in Inguinal Hernia Repair—AI-Based Confirmatory Baseline and Exploration of Competitive Models

DeepPhase: Surgical Phase Recognition in CATARACTS Videos

Towards Deep Learning Guided Autonomous Eye Surgery Using Microscope and iOCT Images

An artificial intelligence platform for the screening and managing of strabismus

Surgical gesture classification from video and kinematic data

Deep learning-based surgical step recognition for laparoscopic right-sided colectomy

Quantification of Robotic Surgeries with Vision-Based Deep Learning

Hierarchical Semi-Supervised Learning Framework for Surgical Gesture Segmentation and Recognition Based on Multi-Modality Data

GLSFormer: Gated - Long, Short Sequence Transformer for Step Recognition in Surgical Videos

Integrating artificial intelligence in strabismus management: current research landscape and future directions

Dual-stage semantic segmentation of endoscopic surgical instruments

Towards Robust Algorithms for Surgical Phase Recognition via Digital Twin-based Scene Representation