Regulating AI Adaptation: An Analysis of AI Medical Device Updates

Kevin Wu,Eric Wu,Kit Rodolfa,Daniel E. Ho,James Zou

DOI: https://doi.org/10.1101/2024.06.26.24309506

2024-06-28

Abstract:While the pace of development of AI has rapidly progressed in recent years, the implementation of safe and effective regulatory frameworks has lagged behind. In particular, the adaptive nature of AI models presents unique challenges to regulators as updating a model can improve its performance but also introduce safety risks. In the US, the Food and Drug Administration (FDA) has been a forerunner in regulating and approving hundreds of AI medical devices. To better understand how AI is updated and its regulatory considerations, we systematically analyze the frequency and nature of updates in FDA-approved AI medical devices. We find that less than 2% of all devices report having been updated by being re-trained on new data. Meanwhile, nearly a quarter of devices report updates in the form of new functionality and marketing claims. As an illustrative case study, we analyze pneumothorax detection models and find that while model performance can degrade by as much as 0.18 AUC when evaluated on new sites, re-training on site-specific data can mitigate this performance drop, recovering up to 0.23 AUC. However, we also observed significant degradation on the original site after re-training using data from new sites, providing insight from one example that challenges the current one-model-fits-all approach to regulatory approvals. Our analysis provides an in-depth look at the current state of FDA-approved AI device updates and insights for future regulatory policies toward model updating and adaptive AI.

Health Informatics

What problem does this paper attempt to address?

This paper mainly discusses the issue of adaptive updates of artificial intelligence (AI) in medical devices, particularly focusing on AI medical devices approved by the U.S. Food and Drug Administration (FDA). The study found that although AI technology is rapidly advancing, the relevant regulatory framework for safety and effectiveness is relatively lagging behind. The adaptive characteristics of AI models present challenges for regulation, as model updates may improve performance but also introduce safety risks.

The paper analyzes the frequency and nature of updates for FDA-approved AI medical devices, and found that less than 2% of devices reported updates due to retraining with new data, while nearly a quarter of devices underwent updates related to functionality or marketing claims. Taking the example of a pulmonary pneumothorax detection model, the study found that the performance of the model in a new environment may be reduced, but retraining on specific site data can improve this decline while also potentially resulting in significant deterioration of the original site performance. This indicates limitations in the current integrated model approval approach.

The paper also mentions that the FDA has started taking measures such as Pre-Specified Change Control Plan (PCCP) to allow limited model changes predetermined in initial approval, but there still exist regulatory challenges regarding AI model updates. The authors call for the future development of more adaptive regulatory policies to address the issues of AI model updates.

In addition, the research found that device updates typically occur within 17 months after approval, and approximately one-third of devices undergo at least one update during their lifecycle. However, the reporting rate of model retraining is much lower than other types of updates.

Overall, this paper reveals the current status of AI medical device updates and provides insights for future development of more flexible and secure regulatory strategies.