Abstract:The hybrid architecture of convolution neural networks (CNN) and Transformer has been the most popular method for medical image segmentation. However, the existing networks based on the hybrid architecture suffer from two problems. First, although the CNN branch can capture image local features by using convolution operation, the vanilla convolution is unable to achieve adaptive extraction of image features. Second, although the Transformer branch can model the global information of images, the conventional self-attention only focuses on the spatial self-attention of images and ignores the channel and cross-dimensional self-attention leading to low segmentation accuracy for medical images with complex backgrounds. To solve these problems, we propose vision Transformer embrace convolutional neural networks for medical image segmentation (TEC-Net). Our network has two advantages. First, dynamic deformable convolution (DDConv) is designed in the CNN branch, which not only overcomes the difficulty of adaptive feature extraction using fixed-size convolution kernels, but also solves the defect that different inputs share the same convolution kernel parameters, effectively improving the feature expression ability of CNN branch. Second, in the Transformer branch, a (shifted)-window adaptive complementary attention module ((S)W-ACAM) and compact convolutional projection are designed to enable the network to fully learn the cross-dimensional long-range dependency of medical images with few parameters and calculations. Experimental results show that the proposed TEC-Net provides better medical image segmentation results than SOTA methods including CNN and Transformer networks. In addition, our TEC-Net requires fewer parameters and computational costs and does not rely on pre-training. The code is publicly available at <a class="link-external link-https" href="https://github.com/SR0920/TEC-Net" rel="external noopener nofollow">this https URL</a>.

TCMINet: Face Parsing for Traditional Chinese Medicine Inspection via a Hybrid Neural Network With Context Aggregation

TCMChat: A Generative Large Language Model for Traditional Chinese Medicine

Toward High Quality Facial Representation Learning

TAMM: A Task-Adaptive Multi-Modal Fusion Network for Facial-Related Health Assessments on 3D Facial Images

End-to-end Face Parsing Via Interlinked Convolutional Neural Networks

Mmy-net: a Multimodal Network Exploiting Image and Patient Metadata for Simultaneous Segmentation and Diagnosis

An Image-Based Facial Acupoint Detection Approach Using High-Resolution Network and Attention Fusion

HTC-Net: A hybrid CNN-transformer framework for medical image segmentation

Research on Data Analysis Network of TCM Tongue Diagnosis Based on Deep Learning Technology

CCNNet: a novel lightweight convolutional neural network and its application in traditional Chinese medicine recognition

Real-time Segmentation and Facial Skin Tones Grading

Face Parsing with RoI Tanh-Warping

Automatic Tongue Image Segmentation for Real-Time Remote Diagnosis

TEC-Net: Vision Transformer Embrace Convolutional Neural Networks for Medical Image Segmentation

MTANet: Multi-Task Attention Network for Automatic Medical Image Segmentation and Classification

Occlusion-Aware Deep Convolutional Neural Network via Homogeneous Tanh-transforms for Face Parsing

A multimodal deep architecture for traditional Chinese medicine diagnosis

CLRNet: Component-Level Refinement Network for Deep Face Parsing

Research and application of tongue and face diagnosis based on deep learning

Automatic Tongue Crack Extraction For Real-Time Diagnosis

Few Shot Medical Image Segmentation with Cross Attention Transformer