Abstract:Colonoscopy is considered the best diagnostic tool for early detection and resection of polyps, which can effectively prevent consequential colorectal cancer. In clinical practice, segmenting and classifying polyps from colonoscopic images have a great significance since they provide precious information for diagnosis and treatment. In this study, we propose an efficient multi-task synergetic network (EMTS-Net) for concurrent polyp segmentation and classification, and we introduce a polyp classification benchmark for exploring the potential correlations of the above-mentioned two tasks. This framework is composed of an enhanced multi-scale network (EMS-Net) for coarse-grained polyp segmentation, an EMTS-Net (Class) for accurate polyp classification, and an EMTS-Net (Seg) for fine-grained polyp segmentation. Specifically, we first obtain coarse segmentation masks by using EMS-Net. Then, we concatenate these rough masks with colonoscopic images to assist EMTS-Net (Class) in locating and classifying polyps precisely. To further enhance the segmentation performance of polyps, we propose a random multi-scale (RMS) training strategy to eliminate the interference caused by redundant information. In addition, we design an offline dynamic class activation mapping (OFLD CAM) generated by the combined effect of EMTS-Net (Class) and RMS strategy, which optimizes bottlenecks between multi-task networks efficiently and elegantly and helps EMTS-Net (Seg) to perform more accurate polyp segmentation. We evaluate the proposed EMTS-Net on the polyp segmentation and classification benchmarks, and it achieves an average mDice of 0.864 in polyp segmentation and an average AUC of 0.913 with an average accuracy of 0.924 in polyp classification. Quantitative and qualitative evaluations on the polyp segmentation and classification benchmarks demonstrate that our EMTS-Net achieves the best performance and outperforms previous state-of-the-art methods in terms of both efficiency and generalization.

MSB-Net: Multi-Scale Boundary Net for Polyp Segmentation

BDG-Net: Boundary Distribution Guided Network for Accurate Polyp Segmentation

MCSF-Net: a multi-scale channel spatial fusion network for real-time polyp segmentation

MCE-Net: polyp segmentation with multiple branch series-parallel attention and channel interaction via edge distribution guidance

Multi-scale Information Sharing and Selection Network with Boundary Attention for Polyp Segmentation

Multi-scale and Multi-path Cascaded Convolutional Network for Semantic Segmentation of Colorectal Polyps

Multi-Dimensional Fusion Reverse Attention Network for Polyp Segmentation

DBE-Net: Dual Boundary-Guided Attention Exploration Network for Polyp Segmentation

Boundary Refinement Network for Colorectal Polyp Segmentation in Colonoscopy Images

ECTransNet: An Automatic Polyp Segmentation Network Based on Multi-scale Edge Complementary

ABC-Net: Area-Boundary Constraint Network With Dynamical Feature Selection for Colorectal Polyp Segmentation

ColonNet: A novel polyp segmentation framework based on LK-RFB and GPPD

MBDA-Net: Multi-source Boundary-Aware Prototype Alignment Domain Adaptation for Polyp Segmentation

M3ResU-Net: a Deep Residual Network for Multi-Center Colorectal Polyp Segmentation Based on Multi-Scale Learning and Attention Mechanism.

Simple and robust depth-wise cascaded network for polyp segmentation

SSN: A Stair-Shape Network for Real-Time Polyp Segmentation in Colonoscopy Images.

An Efficient Multi-Task Synergetic Network for Polyp Segmentation and Classification

Attention based multi-scale parallel network for polyp segmentation

Unveiling camouflaged and partially occluded colorectal polyps: Introducing CPSNet for accurate colon polyp segmentation

Polyp segmentation with distraction separation

MEGANet: Multi-Scale Edge-Guided Attention Network for Weak Boundary Polyp Segmentation