Abstract:Background Recently, many studies have explored fusing features extracted from Convolutional neural networks (CNNs) and transformers to integrate multi‐scale representations for better performance in medical image segmentation tasks. Although these hybrid models have achieved better results than previous CNN‐based and transformer‐based methods, they suffer from high computation and space complexities. Purpose The purpose of this research is to address the prohibitive computation and space complexities of hybrid models, which limit their application in clinical practice where computational resources are usually constrained. Methods We propose a novel model equipped with a dual distillation scheme to sufficiently harness the complementary advantages of CNNs and transformers without compromising model efficiency. We further propose a multi‐scale prior‐knowledge distillation (MPD) module to effectively distill multi‐scale knowledge from features extracted from transformers. In addition, to cooperate with the knowledge distillation scheme, we also propose an efficient and robust Selective Fusion module in the student network. Results We extensively evaluate the proposed model against fourteen different network frameworks on two representative datasets: SipakMed and ISIC 2017. In the SipakMed dataset, 3037 Pap smear images are used for training and 1012 for testing. In the ISIC 2017 dataset, 2000 dermoscopic images are used for training, 150 for validation, and 600 for testing. Experimental results demonstrate that our method not only surpasses existing methods by a considerable margin with respect to the evaluation metrics of mean Intersection over Union, mean Dice coefficient, mean average symmetric surface distance, but also requires fewer computational resources in terms of model parameters and floating‐point operations per second. Conclusions Comprehensive comparisons in terms of segmentation accuracy and computational complexity unequivocally confirm that our method effectively and efficiently integrates the advantages of both CNNs and transformers, showing its suitability and significance for clinical applications.

Dual-stream Representation Fusion Learning for Accurate Medical Image Segmentation

Dual-feature Fusion Attention Network for Small Object Segmentation.

Medical Image Segmentation Based on Adaptive Threshold Prediction and Feature Fusion

DFTNet: Dual-Path Feature Transfer Network for Weakly Supervised Medical Image Segmentation.

A Feature Fusion Module Based on Complementary Attention for Medical Image Segmentation

DCFNet: An Effective Dual-Branch Cross-Attention Fusion Network for Medical Image Segmentation

Unifying and Personalizing Weakly-supervised Federated Medical Image Segmentation via Adaptive Representation and Aggregation

CRMEFNet: A coupled refinement, multiscale exploration and fusion network for medical image segmentation

Region-to-boundary deep learning model with multi-scale feature fusion for medical image segmentation

Dual-Attention Frequency Fusion at Multi-Scale for Joint Segmentation and Deformable Medical Image Registration

End-to-end Dynamic Residual Focal Transformer Network for Multimodal Medical Image Fusion

FedATA: Adaptive Attention Aggregation for Federated Self-supervised Medical Image Segmentation

pFLFE: Cross-silo Personalized Federated Learning via Feature Enhancement on Medical Image Segmentation

A Comprehensive Federated Learning Framework for Diabetic Retinopathy Grading and Lesion Segmentation

Multi‐scale contextual learning for medical image segmentation via dual distillation

Multimodal Fusion Learning with Dual Attention for Medical Imaging

Distributed contrastive learning for medical image segmentation

Dual-stream-based dense local features contrastive learning for semi-supervised medical image segmentation

Semi-Supervised Dual Stream Segmentation Network for Fundus Lesion Segmentation.

Deep Learning-Based Image Segmentation on Multimodal Medical Imaging

DAMAF: dual attention network with multi-level adaptive complementary fusion for medical image segmentation