Abstract:BACKGROUND AND OBJECTIVE:COVID-19 is a serious threat to human health. Traditional convolutional neural networks (CNNs) can realize medical image segmentation, whilst transformers can be used to perform machine vision tasks, because they have a better ability to capture long-range relationships than CNNs. The combination of CNN and transformers to complete the task of semantic segmentation has attracted intense research. Currently, it is challenging to segment medical images on limited data sets like that on COVID-19. METHODS:This study proposes a lightweight transformer+CNN model, in which the encoder sub-network is a two-path design that enables both the global dependence of image features and the low layer spatial details to be effectively captured. Using CNN and MobileViT to jointly extract image features reduces the amount of computation and complexity of the model as well as improves the segmentation performance. So this model is titled Mini-MobileViT-Seg (MMViT-Seg). In addition, a multi query attention (MQA) module is proposed to fuse the multi-scale features from different levels of decoder sub-network, further improving the performance of the model. MQA can simultaneously fuse multi-input, multi-scale low-level feature maps and high-level feature maps as well as conduct end-to-end supervised learning guided by ground truth. RESULTS:The two-class infection labeling experiments were conducted based on three datasets. The final results show that the proposed model has the best performance and the minimum number of parameters among five popular semantic segmentation algorithms. In multi-class infection labeling results, the proposed model also achieved competitive performance. CONCLUSIONS:The proposed MMViT-Seg is tested on three COVID-19 segmentation datasets, with results showing that this model has better performance than other models. In addition, the proposed MQA module, which can effectively fuse multi-scale features of different levels further improves the segmentation accuracy.

CTMLP: Can MLPs replace CNNs or transformers for COVID-19 diagnosis?

MLP-based classification of COVID-19 and skin diseases

MMViT-Seg: A Lightweight Transformer and CNN Fusion Network for COVID-19 Segmentation.

COVID-19 CT image recognition algorithm based on transformer and CNN

An efficient medical image classification network based on multi-branch CNN, token grouping Transformer and mixer MLP

Detection of COVID-19 from Chest CT Images Using CNN with MLP Hybrid Model

An Explainable Non-local Network for COVID-19 Diagnosis

On the differences between CNNs and vision transformers for COVID-19 diagnosis using CT and chest x-ray mono- and multimodality

Using Multi-Task Learning to Improve Diagnostic Performance of Convolutional Neural Networks

Deep learning for COVID-19 detection based on CT images

Explainable Conformer Network for Detection of COVID-19 Pneumonia from Chest CT Scan: From Concepts toward Clinical Explainability

CMM: A CNN-MLP Model for COVID-19 Lesion Segmentation and Severity Grading.

MFL-Net: An Efficient Lightweight Multi-Scale Feature Learning CNN for COVID-19 Diagnosis From CT Images

Multi-DeepNet: A Novel Weakly-Supervised Multi-Task and Multi-View-Oriented Convolution Neural Network for COVID-19 Diagnosis from CT Images

A lightweight CNN-based network on COVID-19 detection using X-ray and CT images

A Comparative Evaluation between Convolutional Neural Networks and Vision Transformers for COVID-19 Detection

Efficient and Visualizable Convolutional Neural Networks for COVID-19 Classification Using Chest CT

Full-resolution MLPs Empower Medical Dense Prediction

Multi-task contrastive learning for automatic CT and X-ray diagnosis of COVID-19

A multi-level feature attention network for COVID-19 detection based on multi-source medical images

Classification of COVID-19 in CT Scans using Multi-Source Transfer Learning