Abstract:Background: Artificial intelligence (AI) medical devices have the potential to transform existing clinical workflows and ultimately improve patient outcomes. AI medical devices have shown potential for a range of clinical tasks such as diagnostics, prognostics, and therapeutic decision-making such as drug dosing. There is, however, an urgent need to ensure that these technologies remain safe for all populations. Recent literature demonstrates the need for rigorous performance error analysis to identify issues such as algorithmic encoding of spurious correlations (eg, protected characteristics) or specific failure modes that may lead to patient harm. Guidelines for reporting on studies that evaluate AI medical devices require the mention of performance error analysis; however, there is still a lack of understanding around how performance errors should be analyzed in clinical studies, and what harms authors should aim to detect and report. Objective: This systematic review will assess the frequency and severity of AI errors and adverse events (AEs) in randomized controlled trials (RCTs) investigating AI medical devices as interventions in clinical settings. The review will also explore how performance errors are analyzed including whether the analysis includes the investigation of subgroup-level outcomes. Methods: This systematic review will identify and select RCTs assessing AI medical devices. Search strategies will be deployed in MEDLINE (Ovid), Embase (Ovid), Cochrane CENTRAL, and clinical trial registries to identify relevant papers. RCTs identified in bibliographic databases will be cross-referenced with clinical trial registries. The primary outcomes of interest are the frequency and severity of AI errors, patient harms, and reported AEs. Quality assessment of RCTs will be based on version 2 of the Cochrane risk-of-bias tool (RoB2). Data analysis will include a comparison of error rates and patient harms between study arms, and a meta-analysis of the rates of patient harm in control versus intervention arms will be conducted if appropriate. Results: The project was registered on PROSPERO in February 2023. Preliminary searches have been completed and the search strategy has been designed in consultation with an information specialist and methodologist. Title and abstract screening started in September 2023. Full-text screening is ongoing and data collection and analysis began in April 2024. Conclusions: Evaluations of AI medical devices have shown promising results; however, reporting of studies has been variable. Detection, analysis, and reporting of performance errors and patient harms is vital to robustly assess the safety of AI medical devices in RCTs. Scoping searches have illustrated that the reporting of harms is variable, often with no mention of AEs. The findings of this systematic review will identify the frequency and severity of AI performance errors and patient harms and generate insights into how errors should be analyzed to account for both overall and subgroup performance. Trial registration: PROSPERO CRD42023387747; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=387747. International registered report identifier (irrid): PRR1-10.2196/51614.

A scoping review of reporting gaps in FDA-approved AI medical devices

Health Disparities and Reporting Gaps in Artificial Intelligence (AI) Enabled Medical Devices: A Scoping Review of 692 U.S. Food and Drug Administration (FDA) 510k Approvals

AI as a Medical Device Adverse Event Reporting in Regulatory Databases: Protocol for a Systematic Review

FDA-Approved Artificial Intelligence and Machine Learning (AI/ML)-Enabled Medical Devices: An Updated Landscape

Promoting Health Equity through Improved Regulation of Artificial Intelligence Medical Devices

Artificial intelligence related safety issues associated with FDA medical device reports

The bias algorithm: how AI in healthcare exacerbates ethnic and racial disparities – a scoping review

Technology-Enabled, Evidence-Driven, and Patient-Centered: The Way Forward for Regulating Software as a Medical Device

Revealing transparency gaps in publicly available COVID-19 datasets used for medical artificial intelligence development—a systematic review

Sources of bias in artificial intelligence that perpetuate healthcare disparities—A global review

Detecting Algorithmic Errors and Patient Harms for AI-Enabled Medical Devices in Randomized Controlled Trials: Protocol for a Systematic Review

AI as a Medical Device for Ophthalmic Imaging in Europe, Australia, and the United States: Protocol for a Systematic Scoping Review of Regulated Devices

Health Equity in Artificial Intelligence and Primary Care Research: Protocol for a Scoping Review

A scoping review of fair machine learning techniques when using real-world data

A Scoping Literature Review on the Ethical Issues Surrounding Artificial Intelligence-Based Medical Devices

Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015–20): a comparative analysis

How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals

Regulating AI Adaptation: An Analysis of AI Medical Device Updates

AI in imaging: the regulatory landscape

Data-Driven Analysis of AI in Medical Device Software in China: Deep Learning and General AI Trends Based on Regulatory Data

Unmasking bias in artificial intelligence: a systematic review of bias detection and mitigation strategies in electronic health record-based models