Abstract:Foundation models hold promise for transforming AI in healthcare by providing modular components that are easily adaptable to downstream healthcare tasks, making AI development more scalable and cost-effective. Structured EHR foundation models, trained on coded medical records from millions of patients, demonstrated benefits including increased performance with fewer training labels, and improved robustness to distribution shifts. However, questions remain on the feasibility of sharing these models across different hospitals and their performance for local task adaptation. This multi-center study examined the adaptability of a recently released structured EHR foundation model ($FM_{SM}$), trained on longitudinal medical record data from 2.57M Stanford Medicine patients. Experiments were conducted using EHR data at The Hospital for Sick Children and MIMIC-IV. We assessed both adaptability via continued pretraining on local data, and task adaptability compared to baselines of training models from scratch at each site, including a local foundation model. We evaluated the performance of these models on 8 clinical prediction tasks. In both datasets, adapting the off-the-shelf $FM_{SM}$ matched the performance of GBM models locally trained on all data while providing a 13% improvement in settings with few task-specific training labels. With continued pretraining on local data, label efficiency substantially improved, such that $FM_{SM}$ required fewer than 1% of training examples to match the fully trained GBM's performance. Continued pretraining was also 60 to 90% more sample-efficient than training local foundation models from scratch. Our findings show that adapting shared EHR foundation models across hospitals provides improved prediction performance at less cost, underscoring the utility of base foundation models as modular components to streamline the development of healthcare AI.

The Shaky Foundations of Clinical Foundation Models: A Survey of Large Language Models and Foundation Models for EMRs

The shaky foundations of large language models and foundation models for electronic health records

A Comprehensive Survey of Foundation Models in Medicine

A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Records

Improving Clinical Expertise in Large Language Models Using Electronic Medical Records

Foundation Models in Radiology: What, How, When, Why and Why Not

Critical Care Studies Using Large Language Models Based on Electronic Healthcare Records: A Technical Note

The new paradigm in machine learning - foundation models, large language models and beyond: a primer for physicians

Foundational Models in Medical Imaging: A Comprehensive Survey and Future Vision

A Clinical Benchmark of Public Self-Supervised Pathology Foundation Models

Foundation Models, Generative AI, and Large Language Models: Essentials for Nursing

When is a Foundation Model a Foundation Model

On the Challenges and Perspectives of Foundation Models for Medical Image Analysis

A Framework for Evaluating the Efficacy of Foundation Embedding Models in Healthcare

Almanac: Retrieval-Augmented Language Models for Clinical Medicine

Foundation models in ophthalmology: opportunities and challenges

Large language models encode clinical knowledge

Medical Multimodal Foundation Models in Clinical Diagnosis and Treatment: Applications, Challenges, and Future Directions

Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation

Evaluation of General Large Language Models in Contextually Assessing Semantic Concepts Extracted from Adult Critical Care Electronic Health Record Notes