Generative AI and large language models in health care: pathways to implementation

Marium M. Raza,Kaushik P. Venkatesh,Joseph C. Kvedar

DOI: https://doi.org/10.1038/s41746-023-00988-4

IF: 15.2

2024-03-08

npj Digital Medicine

Abstract:Generative AI is designed to create new content from trained parameters. Learning from large amounts of data, many of these models aim to simulate human conversation. Generative AI is being applied to many different sectors. Within healthcare there has been innovation specifically towards generative AI models trained on electronic medical record data. A recent review characterizes these models, their strengths, and weaknesses. Inspired by that work, we present our evaluation checklist for generative AI models applied to electronic medical records.

health care sciences & services,medical informatics

What problem does this paper attempt to address?

The paper primarily explores the application and implementation pathways of generative artificial intelligence (AI) and large language models in the healthcare sector. Specifically, the authors focus on how these technologies can be applied to Electronic Medical Records (EMR) data and propose an evaluation framework to assess the effectiveness and practicality of these models.

The paper first introduces the concept of generative AI and its potential application scenarios in healthcare, including simulating human conversations through large language models and writing medical research articles. It then discusses the advantages and limitations of generative AI models in handling EMR data, such as improving predictive performance, simplifying model development processes, and reducing costs, while also highlighting issues like insufficient model generalization and data privacy protection.

To evaluate the actual clinical value of these models, the paper cites a study by Wornow et al., which reviewed 84 foundational models trained on clinical structured text data and proposed an improved framework with six evaluation criteria: predictive performance, data annotation, model deployment, emerging clinical applications, multimodal support, and novel human-computer interaction interfaces. Through this framework, healthcare institutions can better determine which models are most suitable for their specific clinical needs.

Additionally, the paper discusses the leadership, incentive mechanisms, and regulatory measures required to achieve widespread application of generative AI in healthcare. It emphasizes the need for clear leadership to guide model development, validation, and implementation; ongoing regulation to balance the interests of all parties; and appropriate incentive policies to promote the broad adoption of the technology.

In summary, this paper aims to provide a comprehensive perspective on the effective implementation of generative AI in the healthcare sector, including technical evaluation standards and strategic recommendations for driving technological development.