Abstract:Identification of surgical instruments is crucial in understanding surgical scenarios and providing an assistant process in endoscopic image-guided surgery. This study proposes a novel multilevel feature-aggregated deep convolutional neural network (MLFA-Net) for identifying surgical instruments in endoscopic images. First, a global feature augmentation layer is created on the top layer of the backbone to improve the localization ability of object identification by boosting the high-level semantic information to the feature flow network. Second, a modified interaction path of cross-channel features is proposed to increase the nonlinear combination of features in the same level and improve the efficiency of information propagation. Third, a multiview fusion branch of features is built to aggregate the location-sensitive information of the same level in different views, increase the information diversity of features, and enhance the localization ability of objects. By utilizing the latent information, the proposed network of multilevel feature aggregation can accomplish multitask instrument identification with a single network. Three tasks are handled by the proposed network, including object detection, which classifies the type of instrument and locates its border; mask segmentation, which detects the instrument shape; and pose estimation, which detects the keypoint of instrument parts. The experiments are performed on laparoscopic images from MICCAI 2017 Endoscopic Vision Challenge, and the mean average precision (AP) and average recall (AR) are utilized to quantify the segmentation and pose estimation results. For the bounding box regression, the AP and AR are 79.1% and 63.2%, respectively. While the AP and AR of mask segmentation are 78.1% and 62.1%, and the AP and AR of the pose estimation achieve 67.1% and 55.7%, respectively. The experiments demonstrate that our method efficiently improves the recognition accuracy of the instrument in endoscopic images, and outperforms the other state-of-the-art methods.

Surgical Action and Instrument Detection Based on Multiscale Information Fusion

Surgical action detection based on path aggregation adaptive spatial network

Semantic Segmentation of Surgical Instruments Based on Enhanced Multi-scale Receptive Field

Hierarchical Attentional Feature Fusion for Surgical Instrument Segmentation

Application of Feature Pyramid Network and Feature Fusion Single Shot Multibox Detector for Real-Time Prostate Capsule Detection

Attention-based Fusion Factor in FPN for Object Detection

Urtnet: an unstructured feature fusion network for real-time detection of endoscopic surgical instruments

Multi-level feature aggregation network for instrument identification of endoscopic images

EFPN: Effective medical image detection using feature pyramid fusion enhancement

A feature pyramid network with adaptive fusion strategy and enhanced semantic information

Multi-Scale Convolution-Transformer Fusion Network for Endoscopic Image Segmentation.

Image detection of aortic dissection complications based on multi-scale feature fusion

Multi-scale Feature Pyramid Fusion Network for Medical Image Segmentation

SINet: A hybrid deep CNN model for real-time detection and segmentation of surgical instruments

Object Detection in Remote Sensing Images Based on Adaptive Multi-Scale Feature Fusion Method

Multiscale Deformable Attention and Multilevel Features Aggregation for Remote Sensing Object Detection

Expansive Receptive Field and Local Feature Extraction Network: Advancing Multiscale Feature Fusion for Breast Fibroadenoma Segmentation in Sonography

Multiscale Semantic Fusion-Guided Fractal Convolutional Object Detection Network for Optical Remote Sensing Imagery

Multi-Level Medical Image Segmentation Network Based on Multi-Scale and Context Information Fusion Strategy

Dual SIE-FPN: Semantic and Spatial Information Enhancement for Multiscale Object Detection

Cross-Layer Feature Pyramid Network for Salient Object Detection