Abstract:Identification of surgical instruments is crucial in understanding surgical scenarios and providing an assistant process in endoscopic image-guided surgery. This study proposes a novel multilevel feature-aggregated deep convolutional neural network (MLFA-Net) for identifying surgical instruments in endoscopic images. First, a global feature augmentation layer is created on the top layer of the backbone to improve the localization ability of object identification by boosting the high-level semantic information to the feature flow network. Second, a modified interaction path of cross-channel features is proposed to increase the nonlinear combination of features in the same level and improve the efficiency of information propagation. Third, a multiview fusion branch of features is built to aggregate the location-sensitive information of the same level in different views, increase the information diversity of features, and enhance the localization ability of objects. By utilizing the latent information, the proposed network of multilevel feature aggregation can accomplish multitask instrument identification with a single network. Three tasks are handled by the proposed network, including object detection, which classifies the type of instrument and locates its border; mask segmentation, which detects the instrument shape; and pose estimation, which detects the keypoint of instrument parts. The experiments are performed on laparoscopic images from MICCAI 2017 Endoscopic Vision Challenge, and the mean average precision (AP) and average recall (AR) are utilized to quantify the segmentation and pose estimation results. For the bounding box regression, the AP and AR are 79.1% and 63.2%, respectively. While the AP and AR of mask segmentation are 78.1% and 62.1%, and the AP and AR of the pose estimation achieve 67.1% and 55.7%, respectively. The experiments demonstrate that our method efficiently improves the recognition accuracy of the instrument in endoscopic images, and outperforms the other state-of-the-art methods.

A Holistically-Nested U-Net: Surgical Instrument Segmentation Based on Convolutional Neural Network

A Nested U-Structure for Instrument Segmentation in Robotic Surgery

Densely Connected Deep U-Net For Abdominal Multi-Organ Segmentation

PaI-Net: A Modified U-Net of Reducing Semantic Gap for Surgical Instrument Segmentation

I2U-Net: A dual-path U-Net with rich information interaction for medical image segmentation

Attention-Guided Lightweight Network for Real-Time Segmentation of Robotic Surgical Instruments

Towards Better Surgical Instrument Segmentation in Endoscopic Vision: Multi-Angle Feature Aggregation and Contour Supervision

[REPORTS OF THE PHARMACOPOEIAL COMMISSION. 1964, NO. 2].

Dynamic U-Net: Adaptively Calibrate Features for Abdominal Multi-organ Segmentation

A parallel network utilizing local features and global representations for segmentation of surgical instruments

Double-branch U-Net for multi-scale organ segmentation

Rethinking the unpretentious U-net for medical ultrasound image segmentation

DCSAU-Net: A deeper and more compact split-attention U-Net for medical image segmentation

Multi Scale Supervised 3D U-Net for Kidney and Tumor Segmentation

Deep Multi-Scale U-Net Architecture and Label-Noise Robust Training Strategies for Histopathological Image Segmentation

Fast instruments and tissues segmentation of micro-neurosurgical scene using high correlative non-local network

Urtnet: an unstructured feature fusion network for real-time detection of endoscopic surgical instruments

SCU-Net: A Shape-Supervised Contextual-Fusion U-Net for the Dilated Biliary Tree Segmentation.

Multi-level feature aggregation network for instrument identification of endoscopic images

ECU-Net: multi-scale salient boundary detection and contrast feature enhancement U-Net for breast ultrasound image segmentation

SC-UneXt: Nested UNeXt Architecture based on Medical Image Segmentation