Abstract:Background: The launch of the Chat Generative Pre-trained Transformer (ChatGPT) in November 2022 has attracted public attention and academic interest to large language models (LLMs), facilitating the emergence of many other innovative LLMs. These LLMs have been applied in various fields, including healthcare. Numerous studies have since been conducted regarding how to employ state-of-the-art LLMs in health-related scenarios to assist patients, doctors, and public health administrators. Objective: This review aims to summarize the applications and concerns of applying conversational LLMs in healthcare and provide an agenda for future research on LLMs in healthcare. Methods: We utilized PubMed, ACM, and IEEE digital libraries as primary sources for this review. We followed the guidance of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRIMSA) to screen and select peer-reviewed research articles that (1) were related to both healthcare applications and conversational LLMs and (2) were published before September 1st, 2023, the date when we started paper collection and screening. We investigated these papers and classified them according to their applications and concerns. Results: Our search initially identified 820 papers according to targeted keywords, out of which 65 papers met our criteria and were included in the review. The most popular conversational LLM was ChatGPT from OpenAI (60), followed by Bard from Google (1), Large Language Model Meta AI (LLaMA) from Meta (1), and other LLMs (5). These papers were classified into four categories in terms of their applications: 1) summarization, 2) medical knowledge inquiry, 3) prediction, and 4) administration, and four categories of concerns: 1) reliability, 2) bias, 3) privacy, and 4) public acceptability. There are 49 (75%) research papers using LLMs for summarization and/or medical knowledge inquiry, and 58 (89%) research papers expressing concerns about reliability and/or bias. We found that conversational LLMs exhibit promising results in summarization and providing medical knowledge to patients with a relatively high accuracy. However, conversational LLMs like ChatGPT are not able to provide reliable answers to complex health-related tasks that require specialized domain expertise. Additionally, no experiments in our reviewed papers have been conducted to thoughtfully examine how conversational LLMs lead to bias or privacy issues in healthcare research. Conclusions: Future studies should focus on improving the reliability of LLM applications in complex health-related tasks, as well as investigating the mechanisms of how LLM applications brought bias and privacy issues. Considering the vast accessibility of LLMs, legal, social, and technical efforts are all needed to address concerns about LLMs to promote, improve, and regularize the application of LLMs in healthcare.

Based on Medicine, The Now and Future of Large Language Models

Large language model application in emergency medicine and critical care

Large language models in health care: Development, applications, and challenges

A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directions

The future landscape of large language models in medicine

Application Research of Large Language Models in Medicine: Status, Problems, and Future

Critical Care Studies Using Large Language Models Based on Electronic Healthcare Records: A Technical Note

Opportunities, challenges, and future directions of large language models, including ChatGPT in medical education: a systematic scoping review

A Systematic Review of ChatGPT and Other Conversational Large Language Models in Healthcare

A Survey of Large Language Models in Medicine: Progress, Application, and Challenge

The Role of Large Language Models in Medical Education: Applications and Implications

Large Language Models in Medical Education: Opportunities, Challenges, and Future Directions

A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine

A Study of Generative Large Language Model for Medical Research and Healthcare

The application of large language models in medicine: A scoping review

Harnessing the potential of large language models in medical education: promise and pitfalls

Healthcare: A Growing Role for Large Language Models and Generative AI

Large language models: a primer and gastroenterology applications

The Breakthrough of Large Language Models Release for Medical Applications: 1-Year Timeline and Perspectives

Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review

Systematic Review of Large Language Models for Patient Care: Current Applications and Challenges