Abstract:Skin cancer is one of the most serious threats to human health among skin lesions. Computer-aided diagnosis methods can assist patients in identifying and detecting skin lesion types early, thereby enabling corresponding treatments. In this paper, we propose a dual-branch neural network model Conformer with Residual Cosine Similarity Attention and Bidirectional Convolutional fusion strategy, named RCSABC-Conformer. The core of this network structure comprises three parts: a Convolutional Neural Network (CNN) branch with Residual Cosine Similarity Attention (RCSA), a Transformer branch, and a Feature Couple Unit with Bidirectional Convolutional strategy (BC-FCU). The RCSA module calculates the cosine similarity value between the feature map generated by the convolutional operation and the feature map of the residual edge to assess whether their semantic information is similar. The semantic information of similar parts is weighted by exponential normalization to enhance the network's memory of similar features of the same type of skin lesion. The BC-FCU module interactively fuses local features and global representations of skin lesion images with different resolutions in the two branches. Specifically, when the global representations is integrated into local features, we introduce a new bidirectional convolution strategy to extract the feature map from both forward and backward directions, and then select the element with the smaller feature value from the two directions to fuse into local features. In this way, we can minimize the interference of the artifact features extracted by the Transformer branch on the CNN branch. In addition, taking advantage of the Transformer branch's capacity to construct global representations, our model can learn contextual semantic information of normal skin and lesion areas to enhance model robustness. We conducted experiments on three datasets, consisting of clinical and dermoscopic skin lesion images, as well as a hybrid of both. The experimental results show that RCSABC-Conformer outperforms both advanced and classical classification methods in terms of classification accuracy across all three datasets, without requiring an increase in the number of parameters and computational complexity. Compared with the baseline model, the classification accuracy of our proposed method improves by 2.40%, 5.39%, and 4.44% on the three datasets, respectively. To the best of our knowledge, this is the first study to apply an interactive fusion dual-branch network for multi-disease classification on different modalities of skin lesion databases. Code will be available at https://github.com/AlenLi817/RCSABC-Conformer.

MAC-ResNet: Knowledge Distillation Based Lightweight Multiscale-Attention-Crop-ResNet for Eyelid Tumors Detection and Classification.

Deep Learning-Based Accurate Diagnosis of Eyelid Malignant Melanoma from Gigapixel Pathologic Slides

Residual Convolutional Neural Networks With Global And Local Pathways For Classification Of Focal Liver Lesions

A Deep Learning Approach with Cascade-network Design for Eyelid Tumors Diagnosis Based on Gigapixel Histopathology Images

Automated identification of malignancy in whole-slide pathological images: identification of eyelid malignant melanoma in gigapixel pathological slides using deep learning

HM_ADET: a hybrid model for automatic detection of eyelid tumors based on photographic images

Artificial intelligence to detect malignant eyelid tumors from photographic images

Differentiating malignant and benign eyelid lesions using deep learning

Multiple EffNet/ResNet Architectures for Melanoma Classification

A Self-Supervised Learning Based Framework for Eyelid Malignant Melanoma Diagnosis in Whole Slide Images.

A Multiresolution Convolutional Neural Network with Partial Label Training for Annotating Reflectance Confocal Microscopy Images of Skin

Melanoma Segmentation and Classification in Clinical Images Using Deep Learning

A novel approach for automatic classification of macular degeneration OCT images

A New Method Proposed to Melanoma-skin Cancer Lesion Detection and Segmentation Based on Hybrid Convolutional Neural Network

Self-supervised learning mechanism for identification of eyelid malignant melanoma in pathologic slides with limited annotation

ARM-Net: Attention-guided residual multiscale CNN for multiclass brain tumor classification using MR images

Residual Neural Network-Assisted One-Class Classification Algorithm for Melanoma Recognition with Imbalanced Data.

Residual Cosine Similar Attention and Bidirectional Convolution in Dual-Branch Network for Skin Lesion Image Classification

M2LC-Net: A Multi-Modal Multi-Disease Long-Tailed Classification Network for Real Clinical Scenes

Automated deep learning approach for classification of malignant melanoma and benign skin lesions

Melanoma Classification Using a Novel Deep Convolutional Neural Network with Dermoscopic Images