Abstract:Background: Use of artificial intelligence to identify dermoscopic images has brought major breakthroughs in recent years to the early diagnosis and early treatment of skin cancer, the incidence of which is increasing year by year worldwide and poses a great threat to human health. Achievements have been made in the research of skin cancer image classification by using the deep backbone of the convolutional neural network (CNN). This approach, however, only extracts the features of small objects in the image, and cannot locate the important parts. Objectives: As a result, researchers of the paper turn to vision transformers (VIT) which has demonstrated powerful performance in traditional classification tasks. The self-attention is to improve the value of important features and suppress the features that cause noise. Specifically, an improved transformer network named SkinTrans is proposed. Innovations: To verify its efficiency, a three step procedure is followed. Firstly, a VIT network is established to verify the effectiveness of SkinTrans in skin cancer classification. Then multi-scale and overlapping sliding windows are used to serialize the image and multi-scale patch embedding is carried out which pay more attention to multi-scale features. Finally, contrastive learning is used which makes the similar data of skin cancer encode similarly so that the encoding results of different data are as different as possible. Main results: The experiment is carried out based on two datasets, namely (1) HAM10000: a large dataset of multi-source dermatoscopic images of common skin cancers; (2)A clinical dataset of skin cancer collected by dermoscopy. The model proposed has achieved 94.3% accuracy on HAM10000 and 94.1% accuracy on our datasets, which verifies the efficiency of SkinTrans. Conclusions: The transformer network has not only achieved good results in natural language but also achieved ideal results in the field of vision, which also lays a good foundation for skin cancer classification based on multimodal data. This paper is convinced that it will be of interest to dermatologists, clinical researchers, computer scientists and researchers in other related fields, and provide greater convenience for patients.

Breast Tumor Image Classification in Bright Challenge VIA Multiple Instance Learning and Deep Transformers

Transformer based multiple instance learning for WSI breast cancer classification

Breast cancer pathological image classification based on deep learning

Enhancing brain tumor MRI classification with an ensemble of deep learning models and transformer integration

A Multi-Task Transformer with Local-Global Feature Interaction and Multiple Tumoral Region Guidance for Breast Cancer Diagnosis

Attention Multiple Instance Learning with Transformer Aggregation for Breast Cancer Whole Slide Image Classification

Toward High Resolution Scoring System for HER2 Expression in Breast Cancer Based on Pathological Images Via Deep Learning

Transformer-Based Video-Structure Multi-Instance Learning for Whole Slide Image Classification

DESNTC: Transformer-Based Double Shift Network for Breast Histopathological Image Classification

MultiNet with Transformers: A Model for Cancer Diagnosis Using Images

Deep Learning for Identifying Metastatic Breast Cancer

Deep Learning Approaches for Classification of Breast Cancer in Ultrasound (US) Images

Deep Learning-Based Multi-Class Classification of Breast Digital Pathology Images

Breast Ultrasound Tumor Classification Using a Hybrid Multitask CNN-Transformer Network

ViT-CB: Integrating hybrid Vision Transformer and CatBoost to enhanced brain tumor detection with SHAP

A fine-tuned transformer model for brain tumor detection and classification

Ensemble Deep Learning-Based Image Classification for Breast Cancer Subtype and Invasiveness Diagnosis from Whole Slide Image Histopathology

Performance Evaluation of Deep Learning and Transformer Models Using Multimodal Data for Breast Cancer Classification

Breast Cancer Image Classification Method Based on Deep Transfer Learning

Tumor Classification in Automated Breast Ultrasound (ABUS) Based on a Modified Extracting Feature Network

An improved transformer network for skin cancer classification