Abstract:Purpose Breast cancer is the most commonly occurring cancer worldwide. The ultrasound reflectivity imaging technique can be used to obtain breast ultrasound (BUS) images, which can be used to classify benign and malignant tumors. However, the classification is subjective and dependent on the experience and skill of operators and doctors. The automatic classification method can assist doctors and improve the objectivity, but current convolution neural network (CNN) is not good at learning global features and vision transformer (ViT) is not good at extraction local features. In this study, we proposed a visual geometry group attention ViT (VGGA-ViT) network to overcome their disadvantages. Methods In the proposed method, we used a CNN module to extract the local features and employed a ViT module to learn the global relationship among different regions and enhance the relevant local features. The CNN module was named the VGGA module. It was composed of a VGG backbone, a feature extraction fully connected layer, and a squeeze-and-excitation block. Both the VGG backbone and the ViT module were pretrained on the ImageNet dataset and retrained using BUS samples in this study. Two BUS datasets were employed for validation. Results Cross-validation was conducted on two BUS datasets. For the Dataset A, the proposed VGGA-ViT network achieved high accuracy (88.71 +/-$\ \pm \ $1.55%), recall (90.73 +/-$\ \pm \ $1.57%), specificity (85.58 +/-$\ \pm \ $3.35%), precision (90.77 +/-$\ \pm \ $1.98%), F1 score (90.73 +/-$\ \pm \ $1.24%), and Matthews correlation coefficient (MCC) (76.34 +/- 7$\ \pm \ 7$3.29%), which were better than those of all compared previous networks in this study. The Dataset B was used as a separate test set, the test results showed that the VGGA-ViT had highest accuracy (81.72 +/-$\ \pm \ $2.99%), recall (64.45 +/-$\ \pm \ $2.96%), specificity (90.28 +/-$\ \pm \ $3.51%), precision (77.08 +/-$\ \pm \ $7.21%), F1 score (70.11 +/-$\ \pm \ $4.25%), and MCC (57.64 +/-$\ \pm \ $6.88%). Conclusions In this study, we proposed the VGGA-ViT for the BUS classification, which was good at learning both local and global features. The proposed network achieved higher accuracy than the compared previous methods.

Breast Ultrasound Tumor Classification Using a Hybrid Multitask CNN-Transformer Network

Multi-task approach based on combined CNN-transformer for efficient segmentation and classification of breast tumors in ultrasound images

A VGG Attention Vision Transformer Network for Benign and Malignant Classification of Breast Ultrasound Images.

Cross-Attention Based Multi-Scale Feature Fusion Vision Transformer for Breast Ultrasound Image Classification.

Improved breast ultrasound tumor classification using dual-input CNN with GAP-guided attention loss

A Multi-Task Transformer with Local-Global Feature Interaction and Multiple Tumoral Region Guidance for Breast Cancer Diagnosis

Aggregating efficient transformer and CNN networks using learnable fuzzy measure for breast tumor malignancy prediction in ultrasound images

BTS-ST: Swin transformer network for segmentation and classification of multimodality breast cancer images

Breast Tumor Classification in Ultrasound Images by Fusion of Deep Convolutional Neural Network and Shallow LBP Feature

HAU-Net: Hybrid CNN-transformer for breast ultrasound image segmentation

Multi-task learning for segmentation and classification of breast tumors from ultrasound images

HoVer-Trans: Anatomy-aware HoVer-Transformer for ROI-free Breast Cancer Diagnosis in Ultrasound Images

A Multimodal Fusion Model for Breast Tumor Segmentation in Ultrasound Images

Fus2Net: a novel Convolutional Neural Network for classification of benign and malignant breast tumor in ultrasound images

AResNet-ViT: A Hybrid CNN-Transformer Network for Benign and Malignant Breast Nodule Classification in Ultrasound Images

SwinCNN: An Integrated Swin Transformer and CNN for Improved Breast Cancer Grade Classification

CSU-Net: A CNN-Transformer Parallel Network for Multimodal Brain Tumour Segmentation

CSwin-PNet: A CNN-Swin Transformer combined pyramid network for breast lesion segmentation in ultrasound images

A deep supervised transformer U‐shaped full‐resolution residual network for the segmentation of breast ultrasound image

Automatic tumor segmentation in breast ultrasound images using a dilated fully convolutional network combined with an active contour model.

ESTAN: Enhanced Small Tumor-Aware Network for Breast Ultrasound Image Segmentation