Abstract:Background and Objectives: Transformers using self-attention mechanisms have recently advanced medical imaging by modeling long-range semantic dependencies, though they lack CNNs’ ability to capture local spatial details. This study introduced a novel segmentation network derived from a mixed CNN-Transformer (MixFormer) feature extraction backbone to enhance medical image segmentation. Method: The MixFormer network seamlessly integrates global and local information from Transformer and CNN architectures during the downsampling process. To comprehensively capture the inter-scale perspective, we introduced a Multi-scale Spatial-aware Fusion (MSAF) module, enabling effective interaction between coarse and fine feature representations. Additionally, we proposed a Mixed Multi-branch Dilated Attention (MMDA) module to bridge the semantic gap between encoding and decoding stages while emphasizing specific regions. Lastly, we implemented a CNN-based upsampling approach to recover low-level features, substantially improving segmentation accuracy. Results: Experimental validations on prevalent medical image datasets demonstrated the superior performance of MixFormer. On the Synapse dataset, our approach achieved a mean Dice Similarity Coefficient (DSC) of 82.64% and a mean Hausdorff Distance (HD) of 12.67 mm. On the ACDC dataset, the DSC was 91.01%. On the ISIC 2018 dataset, the model achieved a mean Intersection over Union (mIOU) of 0.841, Accuracy of 0.958, Precision of 0.910, Recall of 0.934, and an F1 score of 0.913. For the Kvasir-SEG dataset, we recorded a mean Dice of 0.9247, mIOU of 0.8615, Precision of 0.9181, and Recall of 0.9463. On the CVC-ClinicDB dataset, the results were a mean Dice of 0.9441, mIOU of 0.8922, Precision of 0.9437, and Recall of 0.9458. Conclusion: These findings underscore the superior segmentation performance of MixFormer compared to most mainstream segmentation networks such as CNNs and other Transformerbased structures.

Combined query image retrieval based on hybrid coding of CNN and Mix-Transformer

Aggregating Hierarchical Binary Activations for Image Retrieval

Exploiting Hierarchical Activations of Neural Network for Image Retrieval.

MixFormer: a Mixed CNN-Transformer Backbone for Medical Image Segmentation

Combined query embroidery image retrieval based on enhanced CNN and blend transformer

A Multi-stage Feature Enhancement Network with Second-Attention for Image Captioning

HybridHash: Hybrid Convolutional and Self-Attention Deep Hashing for Image Retrieval

Combining Self-Aggregation Vector and Indicator Matrix for Improved Object Retrieval

Mutual Guidance and Residual Integration for Image Enhancement

Multihead Global Attention and Spatial Spectral Information Fusion for Remote Sensing Image Compression

Single- and Cross-Modality Near Duplicate Image Pairs Detection Via Spatial Transformer Comparing CNN

CTA-Net: A CNN-Transformer Aggregation Network for Improving Multi-Scale Feature Extraction

Multi-branch CNN and grouping cascade attention for medical image classification

Combining Convolutional Neural Network and Markov Random Field for Semantic Image Retrieval

Image Retrieval with Composed Query by Multi-Scale Multi-Modal Fusion.

Exploiting the Complementary Strengths of Multi-Layer CNN Features for Image Retrieval.

Advancing Vision Transformers with Group-Mix Attention

Mutually Guided Learning of Global Semantics and Local Representations for Image Restoration

SMMix: Self-Motivated Image Mixing for Vision Transformers

Hybrid Densely Connected Network for Multi-Exposure Image Fusion

Retrieving Images by Multiple Samples via Fusing Deep Features.