Abstract:Purpose Breast cancer is the most commonly occurring cancer worldwide. The ultrasound reflectivity imaging technique can be used to obtain breast ultrasound (BUS) images, which can be used to classify benign and malignant tumors. However, the classification is subjective and dependent on the experience and skill of operators and doctors. The automatic classification method can assist doctors and improve the objectivity, but current convolution neural network (CNN) is not good at learning global features and vision transformer (ViT) is not good at extraction local features. In this study, we proposed a visual geometry group attention ViT (VGGA-ViT) network to overcome their disadvantages. Methods In the proposed method, we used a CNN module to extract the local features and employed a ViT module to learn the global relationship among different regions and enhance the relevant local features. The CNN module was named the VGGA module. It was composed of a VGG backbone, a feature extraction fully connected layer, and a squeeze-and-excitation block. Both the VGG backbone and the ViT module were pretrained on the ImageNet dataset and retrained using BUS samples in this study. Two BUS datasets were employed for validation. Results Cross-validation was conducted on two BUS datasets. For the Dataset A, the proposed VGGA-ViT network achieved high accuracy (88.71 +/-$\ \pm \ $1.55%), recall (90.73 +/-$\ \pm \ $1.57%), specificity (85.58 +/-$\ \pm \ $3.35%), precision (90.77 +/-$\ \pm \ $1.98%), F1 score (90.73 +/-$\ \pm \ $1.24%), and Matthews correlation coefficient (MCC) (76.34 +/- 7$\ \pm \ 7$3.29%), which were better than those of all compared previous networks in this study. The Dataset B was used as a separate test set, the test results showed that the VGGA-ViT had highest accuracy (81.72 +/-$\ \pm \ $2.99%), recall (64.45 +/-$\ \pm \ $2.96%), specificity (90.28 +/-$\ \pm \ $3.51%), precision (77.08 +/-$\ \pm \ $7.21%), F1 score (70.11 +/-$\ \pm \ $4.25%), and MCC (57.64 +/-$\ \pm \ $6.88%). Conclusions In this study, we proposed the VGGA-ViT for the BUS classification, which was good at learning both local and global features. The proposed network achieved higher accuracy than the compared previous methods.

Multi-task approach based on combined CNN-transformer for efficient segmentation and classification of breast tumors in ultrasound images

Breast Ultrasound Tumor Classification Using a Hybrid Multitask CNN-Transformer Network

Mmformer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentation

Cross-Attention Based Multi-Scale Feature Fusion Vision Transformer for Breast Ultrasound Image Classification.

A Multi-Task Transformer with Local-Global Feature Interaction and Multiple Tumoral Region Guidance for Breast Cancer Diagnosis

Aggregating efficient transformer and CNN networks using learnable fuzzy measure for breast tumor malignancy prediction in ultrasound images

EfficientUNetViT: Efficient Breast Tumor Segmentation Utilizing UNet Architecture and Pretrained Vision Transformer

A VGG Attention Vision Transformer Network for Benign and Malignant Classification of Breast Ultrasound Images.

A Multi-Task Learning Framework for Automated Segmentation and Classification of Breast Tumors From Ultrasound Images

Multi-task learning for segmentation and classification of breast tumors from ultrasound images

MFMSNet: A Multi-frequency and Multi-scale Interactive CNN-Transformer Hybrid Network for breast ultrasound image segmentation

A dual-stage transformer and MLP-based network for breast ultrasound image segmentation

HAU-Net: Hybrid CNN-transformer for breast ultrasound image segmentation

AResNet-ViT: A Hybrid CNN-Transformer Network for Benign and Malignant Breast Nodule Classification in Ultrasound Images

Vision Transformer for Classification of Breast Ultrasound Images

BUViTNet: Breast Ultrasound Detection via Vision Transformers

Combining the Transformer and Convolution for Effective Brain Tumor Classification Using MRI Images

Vision transformer-convolution for breast cancer classification using mammography images: A comparative study

Breast Tumor Classification in Ultrasound Images by Fusion of Deep Convolutional Neural Network and Shallow LBP Feature

Multi-task Learning for Segmentation and Classification of Tumors in 3D Automated Breast Ultrasound Images

MMMViT: Multiscale multimodal vision transformer for brain tumor segmentation with missing modalities