Abstract:Machine learning (ML) methods are becoming more prevalent in the neurology literature as alternatives to traditional statistical methods to address challenges in the analysis of modern data sets. Despite the increase in the popularity of ML methods in neurology studies, some authors do not fully address all items recommended in reporting guidelines. The authors of this Research Methods article are members of the Neurology® editorial board and have reviewed many studies using ML methods. In their review reports, several critiques often appear, which could be avoided if guidance were available. In this article, we detail common critiques found in ML research studies and make recommendations for how to avoid them. The first critique involves misalignment of the study goals and the analysis conducted. The second critique focuses on ML terminology being appropriately used. Critiques 3-6 are related to the study design: justifying sample sizes and the suitability of the data set for the study goals, describing the ML analysis pipeline sufficiently, quantifying the amount of missing data and providing information about missing data handling, and including uncertainty estimates for key metrics. The seventh critique focuses on fairly describing both strengths and limitations of the ML study, including the analysis methodology and results. We provide examples in neurology for each critique and guidance on how to avoid the critique. Overall, we recommend that authors use ML-specific checklists developed by research consortia for designing and reporting studies using ML. We also recommend that authors involve both a statistician and an ML expert in work that uses ML. Although our list of critiques is not exhaustive, our recommendations should help improve the quality and rigor of ML studies. ML has great potential to revolutionize neurology, but investigators need to conduct and report the results in a way that allows readers to fully evaluate the benefits and limitations of ML approaches.

A Guide for the Application of Statistics in Biomedical Studies Concerning Machine Learning and Artificial Intelligence

Statistical Thinking, Machine Learning

An Overview of Machine Learning in Orthopedic Surgery: An Educational Paper

A clinician's guide to understanding and critically appraising machine learning studies: a checklist for Ruling Out Bias Using Standard Tools in Machine Learning (ROBUST-ML)

Common Critiques and Recommendations for Studies in Neurology Using Machine Learning Methods

A survey of machine learning techniques in medical applications

A Comprehensive Review of the Application of Machine Learning in Medicine and Health Care

Recommended Requirements and Essential Elements for Proper Reporting of the Use of Artificial Intelligence Machine Learning Tools in Biomedical Research and Scientific Publications

Supervised machine learning and associated algorithms: applications in orthopedic surgery

AI/ML in Precision Medicine: A Look Beyond the Hype

Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness

Machine learning prediction models in orthopedic surgery: A systematic review in transparent reporting

Meaningless Applications and Misguided Methodologies in Artificial Intelligence-Related Orthopaedic Research Propagates Hype Over Hope

An introduction to machine learning and analysis of its use in rheumatic diseases

The Use of Artificial Intelligence in Orthopedics: Applications and Limitations of Machine Learning in Diagnosis and Prediction

Machine learning in medicine: a practical introduction to techniques for data pre-processing, hyperparameter tuning, and model comparison

Clinical Applications of Machine Learning

A practical guide to the implementation of artificial intelligence in orthopaedic research-Part 2: A technical introduction

Machine Learning Application in Genomic, Exercise, and Vital Datasets

Editorial Commentary: Machine Learning and Artificial Intelligence Are Valuable Tools yet Dependent on the Data Input

An Overview of Machine Learning Applications in Sports Injury Prediction