Abstract:Introduction: Automated machine learning (autoML) removes technical and technological barriers to building artificial intelligence models. We aimed to summarise the clinical applications of autoML, assess the capabilities of utilised platforms, evaluate the quality of the evidence trialling autoML, and gauge the performance of autoML platforms relative to conventionally developed models, as well as each other. Method: This review adhered to a prospectively registered protocol (PROSPERO identifier CRD42022344427). The Cochrane Library, Embase, MEDLINE and Scopus were searched from inception to 11 July 2022. Two researchers screened abstracts and full texts, extracted data and conducted quality assessment. Disagreement was resolved through discussion and as if required, arbitration by a third researcher. Results: There were 26 distinct autoML platforms featured in 82 studies. Brain and lung disease were the most common fields of study of 22 specialties. AutoML exhibited variable performance: area under the receiver operator characteristic curve (AUCROC) 0.35–1.00, F1-score 0.16–0.99, area under the precision-recall curve (AUPRC) 0.51–1.00. AutoML exhibited the highest AUCROC in 75.6% trials; the highest F1-score in 42.3% trials; and the highest AUPRC in 83.3% trials. In autoML platform comparisons, AutoPrognosis and Amazon Rekognition performed strongest with unstructured and structured data, respectively. Quality of reporting was poor, with a median DECIDE-AI score of 14 of 27. Conclusion: A myriad of autoML platforms have been applied in a variety of clinical contexts. The performance of autoML compares well to bespoke computational and clinical benchmarks. Further work is required to improve the quality of validation studies. AutoML may facilitate a transition to data-centric development, and integration with large language models may enable AI to build itself to fulfil user-defined goals.

Bridging expertise with machine learning and automated machine learning in clinical medicine

Clinical performance of automated machine learning: A systematic review

Machine learning in health care and laboratory medicine: General overview of supervised learning and Auto-ML

Machine learning in medicine: what clinicians should know

Automated Machine Learning for Healthcare and Clinical Notes Analysis

Democratizing Artificial Intelligence Imaging Analysis With Automated Machine Learning: Tutorial

Applied machine learning in cancer research: A systematic review for patient diagnosis, classification and prognosis

Strategies for Implementing Machine Learning Algorithms in the Clinical Practice of Radiology

Addressing the Challenges and Barriers to the Integration of Machine Learning into Clinical Practice: An Innovative Method to Hybrid Human–Machine Intelligence

Systematic reviews of machine learning in healthcare: a literature review

Clinical Applications of Machine Learning

Unlocking the Potential of Artificial Intelligence in Acute Myeloid Leukemia and Myelodysplastic Syndromes

A narrative review of machine learning as promising revolution in clinical practice of scoliosis

Application of Machine Learning in Chronic Kidney Disease: Current Status and Future Prospects

Automated machine learning with R: AutoML tools for beginners in clinical research

Advancements in medical diagnosis and treatment through machine learning: A review

Machine learning and artificial intelligence: applications in healthcare epidemiology

AutoML Systems For Medical Imaging

Machine learning in medicine: a practical introduction to techniques for data pre-processing, hyperparameter tuning, and model comparison

Human-machine teaming is key to AI adoption: clinicians' experiences with a deployed machine learning system