Abstract:Accurate classification and segmentation of polyps are two important tasks in the diagnosis and treatment of colorectal cancers. Existing models perform segmentation and classification separately and do not fully make use of the correlation between the two tasks. Furthermore, polyps exhibit random regions and varying shapes and sizes, and they often share similar boundaries and backgrounds. However, existing models fail to consider these factors and thus are not robust because of their inherent limitations. To address these issues, we developed a multi-task network that performs both segmentation and classification simultaneously and can cope with the aforementioned factors effectively. Our proposed network possesses a dual-branch structure, comprising a transformer branch and a convolutional neural network (CNN) branch. This approach enhances local details within the global representation, improving both local feature awareness and global contextual understanding, thus contributing to the improved preservation of polyp-related information. Additionally, we have designed a feature interaction module (FIM) aimed at bridging the semantic gap between the two branches and facilitating the integration of diverse semantic information from both branches. This integration enables the full capture of global context information and local details related to polyps. To prevent the loss of edge detail information crucial for polyp identification, we have introduced a reverse attention boundary enhancement (RABE) module to gradually enhance edge structures and detailed information within polyp regions. Finally, we conducted extensive experiments on five publicly available datasets to evaluate the performance of our method in both polyp segmentation and classification tasks. The experimental results confirm that our proposed method outperforms other state-of-the-art methods.

Single-Modality Endoscopic Polyp Segmentation Via Random Color Reversal Synthesis and Two-Branched Learning

Probabilistic Modeling Ensemble Vision Transformer Improves Complex Polyp Segmentation

SSN: A Stair-Shape Network for Real-Time Polyp Segmentation in Colonoscopy Images.

Simultaneous segmentation and classification of colon cancer polyp images using a dual branch multi-task learning network

Dual-Domain Learning Network for Polyp Segmentation

SSTFB: Leveraging self-supervised pretext learning and temporal self-attention with feature branching for real-time video polyp segmentation

Adaptive Context Selection for Polyp Segmentation

PSTNet: Enhanced Polyp Segmentation with Multi-scale Alignment and Frequency Domain Integration

Dual encoder–decoder-based deep polyp segmentation network for colonoscopy images

Self-Supervised and Semi-Supervised Polyp Segmentation using Synthetic Data

Joint one-sided synthetic unpaired image translation and segmentation for colorectal cancer prevention

Multi-Dimensional Fusion Reverse Attention Network for Polyp Segmentation

RetSeg: Retention-based Colorectal Polyps Segmentation Network

ECTransNet: An Automatic Polyp Segmentation Network Based on Multi-scale Edge Complementary

Three-stage polyp segmentation network based on reverse attention feature purification with Pyramid Vision Transformer

Shallow Attention Network for Polyp Segmentation

Non-equivalent images and pixels: Confidence-aware resampling with meta-learning mixup for polyp segmentation

Colorectal Polyp Segmentation by U-Net with Dilation Convolution

SegT: A Novel Separated Edge-guidance Transformer Network for Polyp Segmentation

SRSegNet: Super-resolution-assisted small targets polyp segmentation network with combined high and low resolution

Colorectal polyp segmentation with denoising diffusion probabilistic models