Abstract:Automatic segmentation and classification of lesions are two clinically significant tasks in the computer-aided diagnosis of skin diseases. Both tasks are challenging due to the nonnegligible lesion differences in dermoscopic images from different patients. In this paper, we propose a novel pipeline to efficiently implement skin lesions' segmentation and classification tasks, which consists of a segmentation network and a classification network. To improve the performance of the segmentation network, we propose a novel module of Multi -Scale Holistic Feature Exploration (MSH) to thoroughly exploit perceptual clues latent among multi-scale feature maps as synthesized by the decoder. The MSH module enables holistic exploration of features across multiple scales to more effectively support downstream image analysis tasks. To boost the performance of the classification network, we propose a novel module of Cross-Modality Collaborative Feature Exploration (CMC) to discover latent discriminative features by collaboratively exploiting potential relationships between cross -modal features of dermoscopic images and clinical metadata. The CMC module enables dynamically capturing versatile interaction effects among cross-modal features during the model's representation learning procedure by discriminatively and adaptively learning the interaction weight associated with each crossmodality feature pair. In addition, to effectively reduce background noise and boost the lesion discrimination ability of the classification network, we crop the images based on lesion masks generated by the best segmentation model. We evaluate the proposed pipeline on the four public skin lesion datasets, where the ISIC 2018 and PH2 are for segmentation, and the ISIC 2019 and ISIC 2020 are combined into a new dataset, ISIC 2019&2020, for classification. It achieves a Jaccard index of 83.31% and 90.14% in skin lesion segmentation, an AUC of 97.98% and an Accuracy of 92.63% in skin lesion classification, which is superior to the performance of representative state-of-the-art skin lesion segmentation and classification methods. Last but not least, the new model for segmentation utilizes much fewer model parameters (3.3 M) than its peer approaches, leading to a greatly reduced number of labeled samples required for model training, which obtains substantially stronger robustness than its peers.

An Unsupervised Multispectral Image Registration Network for Skin Diseases

Intersection-union dual-stream cross-attention Lova-SwinUnet for skin cancer hair segmentation and image repair

Learning from Dermoscopic Images in Association with Clinical Metadata for Skin Lesion Segmentation and Classification

MSS-UNet: A Multi-Spatial-Shift MLP-based UNet for skin lesion segmentation

Convolutional neural network based on regional features and dimension matching for skin cancer classification

Multispectral Snapshot Image Registration Using Learned Cross Spectral Disparity Estimation and a Deep Guided Occlusion Reconstruction Network

Attention-Guided Learning with Feature Reconstruction for Skin Lesion Diagnosis using Clinical and Ultrasound Images

Automatic melanoma detection via multi-scale lesion-biased representation and joint reverse classification

Unsupervised Registration Method based on Deep Neural Network: Application to cardiac and liver MR images

Multimodal Remote Sensing Image Registration Based on Adaptive Spectrum Congruency

Unsupervised Registration for Liver CT-MR Images Based on the Multiscale Integrated Spatial-Weight Module and Dual Similarity Guidance.

A Multi-Stage Framework With Context Information Fusion Structure For Skin Lesion Segmentation

Weakly Supervised Registration of Prostate MRI and Histopathology Images

SG-MIAN: Self-guided multiple information aggregation network for image-level weakly supervised skin lesion segmentation

Unsupervised MMRegNet Based on Spatially Encoded Gradient Information.

Exploring Web Images to Enhance Skin Disease Analysis under A Computer Vision Framework

Interpretability-Based Multimodal Convolutional Neural Networks for Skin Lesion Diagnosis

AI-Skin : Skin Disease Recognition based on Self-learning and Wide Data Collection through a Closed Loop Framework

An End-to-End Multi-Task Deep Learning Framework for Skin Lesion Analysis

Multi-level Relationship Capture Network for Automated Skin Lesion Recognition

Self-Supervised Multi-Modality Learning for Multi-Label Skin Lesion Classification