Abstract:Background: Deep-learning convolutional neural networks (CNNs) have outperformed even experienced dermatologists in dermoscopic melanoma detection under controlled conditions. It remains unexplored how real-world dermoscopic image transformations affect CNN robustness. Objectives: To investigate the consistency of melanoma risk assessment by two commercially available CNNs to help formulate recommendations for current clinical use. Methods: A comparative cohort study was conducted from January to July 2022 at the Department of Dermatology, University Hospital Basel. Five dermoscopic images of 116 different lesions on the torso of 66 patients were captured consecutively by the same operator without deliberate rotation. Classification was performed by two CNNs (CNN-1/CNN-2). Lesions were divided into four subgroups based on their initial risk scoring and clinical dignity assessment. Reliability was assessed by variation and intraclass correlation coefficients. Excisions were performed for melanoma suspicion or two consecutively elevated CNN risk scores, and benign lesions were confirmed by expert consensus (n = 3). Results: 117 repeated image series of 116 melanocytic lesions (2 melanomas, 16 dysplastic naevi, 29 naevi, 1 solar lentigo, 1 suspicious and 67 benign) were classified. CNN-1 demonstrated superior measurement repeatability for clinically benign lesions with an initial malignant risk score (mean variation coefficient (mvc): CNN-1: 49.5(±34.3)%; CNN-2: 71.4(±22.5)%; p = 0.03), while CNN-2 outperformed for clinically benign lesions with benign scoring (mvc: CNN-1: 49.7(±22.7)%; CNN-2: 23.8(±29.3)%; p = 0.002). Both systems exhibited lowest score consistency for lesions with an initial malignant risk score and benign assessment. In this context, averaging three initial risk scores achieved highest sensitivity of dignity assessment (CNN-1: 94%; CNN-2: 89%). Intraclass correlation coefficients indicated 'moderate'-to-'good' reliability for both systems (CNN-1: 0.80, 95% CI:0.71-0.87, p < 0.001; CNN-2: 0.67, 95% CI:0.55-0.77, p < 0.001). Conclusions: Potential user-induced image changes can significantly influence CNN classification. For clinical application, we recommend using the average of three initial risk scores. Furthermore, we advocate for CNN robustness optimization by cross-validation with repeated image sets. Trial registration: ClinicalTrials.gov (NCT04605822).

Analytic validation of convolutional neural network-based classification of pigmented skin lesions (PSLs) using unseen PSL hyperspectral data for clinical applications

Computerizing the first step of the two-step algorithm in dermoscopy: A convolutional neural network for differentiating melanocytic from non-melanocytic skin lesions

Automatic identification of benign pigmented skin lesions from clinical images using deep convolutional neural network

Systematic literature review of dermoscopic pigmented skin lesions classification using convolutional neural network (CNN)

Consistency of convolutional neural networks in dermoscopic melanoma recognition: A prospective real-world study about the pitfalls of augmented intelligence

Expert-Level Diagnosis of Nonpigmented Skin Cancer by Combined Convolutional Neural Networks

A deep convolutional neural network-based pigmented skin lesion classification application and experts evaluation

Clinical performance of a novel hyperspectral imaging device for cutaneous melanoma and pigmented skin lesions in Caucasian skin

Using computational learning for non-melanoma skin cancer and actinic keratosis near-infrared hyperspectral signature classification

Interpretable Classification from Skin Cancer Histology Slides Using Deep Learning: A Retrospective Multicenter Study

Comprehensive analysis of clinical images contributions for melanoma classification using convolutional neural networks

A novel CNN framework for skin disease classification using adaptive percentage filter for image binarization and fast-marching inpainting method

Image classification and auxiliary diagnosis system for hyperpigmented skin diseases based on deep learning

Automatic detection of head and neck squamous cell carcinoma on pathologic slides using polarized hyperspectral imaging and deep learning

Dark corner artefact and diagnostic performance of a market‐approved neural network for skin cancer classification

Evaluation of Big Data based CNN Models in Classification of Skin Lesions with Melanoma

Integrating color histogram analysis and convolutional neural networks for skin lesion classification

An approach to the dermatological classification of histopathological skin images using a hybridized CNN-DenseNet model

Pixel-level classification of pigmented skin cancer lesions using multispectral autofluorescence lifetime dermoscopy imaging

Light-Dermo: A Lightweight Pretrained Convolution Neural Network for the Diagnosis of Multiclass Skin Lesions

Siamese Neural Networks for Skin Cancer Classification and New Class Detection using Clinical and Dermoscopic Image Datasets