Abstract:Background A number of studies have shown promising performance of artificial intelligence (AI) algorithms for diagnosis of lesions in skin cancer. To date, none of these have assessed algorithm performance in the real-world setting. Objective The aim of this project is to evaluate practical issues of implementing a convolutional neural network developed by MoleMap Ltd and Monash University eResearch in the clinical setting. Methods Participants were recruited from the Alfred Hospital and Skin Health Institute, Melbourne, Australia, from November 1, 2019, to May 30, 2021. Any skin lesions of concern and at least two additional lesions were imaged using a proprietary dermoscopic camera. Images were uploaded directly to the study database by the research nurse via a custom interface installed on a clinic laptop. Doctors recorded their diagnosis and management plan for each lesion in real time. A pre-post study design was used. In the preintervention period, participating doctors were blinded to AI lesion assessment. An interim safety analysis for AI accuracy was then performed. In the postintervention period, the AI algorithm classified lesions as benign, malignant, or uncertain after the doctors’ initial assessment had been made. Doctors then had the opportunity to record an updated diagnosis and management plan. After discussing the AI diagnosis with the patient, a final management plan was agreed upon. Results Participants at both sites were high risk (for example, having a history of melanoma or being transplant recipients). 743 lesions were imaged in 214 participants. In total, 28 dermatology trainees and 17 consultant dermatologists provided diagnoses and management decisions, and 3 experienced teledermatologists provided remote assessments. A dedicated research nurse was essential to oversee study processes, maintain study documents, and assist with clinical workflow. In cases where AI algorithm and consultant dermatologist diagnoses were discordant, participant anxiety was an important factor in the final agreed management plan to biopsy or not. Conclusions Although AI algorithms are likely to be of most use in the primary care setting, higher event rates in specialist settings are important for the initial assessment of algorithm safety and accuracy. This study highlighted the importance of considering workflow issues and doctor-patient-AI interactions prior to larger-scale trials in community-based practices. Acknowledgments This research was supported by the Victorian Medical Research Acceleration Fund, with 1:1 contribution from MoleMap Ltd. VM is supported by the National Health and Medical Research Council Early Career Fellowship. CF is supported by the Monash University Research Training Program Scholarship. Conflicts of Interest SM is head of clinical research and regulatory affairs at Kahu.ai Ltd, a subsidiary of MoleMap Ltd. MH was the chief medical officer and a director of MoleMap Ltd, and holds shares in MoleMap Ltd. Trial Registration ClinicalTrials.gov NCT04040114; https://clinicaltrials.gov/ct2/show/NCT04040114

Real-world post-deployment performance of a novel machine learning-based digital health technology for skin lesion assessment and suggestions for post-market surveillance

BT09 Clinical performance of an artificial intelligence-based medical device deployed within an urgent suspected skin cancer pathway

BT09 Multicentre prospective clinical performance analysis of an artificial intelligence as a medical device deployed within urgent suspected skin cancer pathways

Artificial Intelligence Support for Skin Lesion Triage in Primary Care and Dermatology (Preprint)

Accuracy of an artificial intelligence as a medical device as part of a UK-based skin cancer teledermatology service

Artificial Intelligence Support for Skin Lesion Triage in Primary Care and Dermatology

Testing Artificial Intelligence Algorithms in the Real World: Lessons From the SMARTI Trial

Teledermatology & Digital Dermatology SymposiumOral presentationsBT01 Developing a national artificial intelligence-assisted skin cancer pathway

Clinical Utility of a Digital Dermoscopy Image-Based Artificial Intelligence Device in the Diagnosis and Management of Skin Cancer by Dermatologists

BT12 Great expectations: implementing artificial intelligence software into a regional skin cancer teledermatology service

Automatized self-supervised learning for skin lesion screening

Patient perspectives of artificial intelligence as a medical device in a skin cancer pathway

Prospective validation of dermoscopy-based open-source artificial intelligence for melanoma diagnosis (PROVE-AI study)

Evaluation of an artificial intelligence-based decision support for detection of cutaneous melanoma in primary care – a prospective, real-life, clinical trial

BT06 Using artificial intelligence to triage skin cancer referrals: outcomes from a pilot study

Performance of Commercial Dermatoscopic Systems That Incorporate Artificial Intelligence for the Identification of Melanoma in General Practice: A Systematic Review

Analysis of Artificial Intelligence-Based Approaches Applied to Non-Invasive Imaging for Early Detection of Melanoma: A Systematic Review

The utility and reliability of a deep learning algorithm as a diagnosis support tool in head & neck non-melanoma skin malignancies

Implementation of artificial intelligence for the detection of cutaneous melanoma within a primary care setting: prevalence and types of skin cancer in outdoor enthusiasts

P042 Developing a clinical audit methodology for monitoring dermatologist performance in artificial intelligence-enabled teledermatology pathways

Learnings from the first AI-enabled skin cancer device for primary care authorized by FDA