Abstract:Background: The launch of the Chat Generative Pre-trained Transformer (ChatGPT) in November 2022 has attracted public attention and academic interest to large language models (LLMs), facilitating the emergence of many other innovative LLMs. These LLMs have been applied in various fields, including healthcare. Numerous studies have since been conducted regarding how to employ state-of-the-art LLMs in health-related scenarios to assist patients, doctors, and public health administrators. Objective: This review aims to summarize the applications and concerns of applying conversational LLMs in healthcare and provide an agenda for future research on LLMs in healthcare. Methods: We utilized PubMed, ACM, and IEEE digital libraries as primary sources for this review. We followed the guidance of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRIMSA) to screen and select peer-reviewed research articles that (1) were related to both healthcare applications and conversational LLMs and (2) were published before September 1st, 2023, the date when we started paper collection and screening. We investigated these papers and classified them according to their applications and concerns. Results: Our search initially identified 820 papers according to targeted keywords, out of which 65 papers met our criteria and were included in the review. The most popular conversational LLM was ChatGPT from OpenAI (60), followed by Bard from Google (1), Large Language Model Meta AI (LLaMA) from Meta (1), and other LLMs (5). These papers were classified into four categories in terms of their applications: 1) summarization, 2) medical knowledge inquiry, 3) prediction, and 4) administration, and four categories of concerns: 1) reliability, 2) bias, 3) privacy, and 4) public acceptability. There are 49 (75%) research papers using LLMs for summarization and/or medical knowledge inquiry, and 58 (89%) research papers expressing concerns about reliability and/or bias. We found that conversational LLMs exhibit promising results in summarization and providing medical knowledge to patients with a relatively high accuracy. However, conversational LLMs like ChatGPT are not able to provide reliable answers to complex health-related tasks that require specialized domain expertise. Additionally, no experiments in our reviewed papers have been conducted to thoughtfully examine how conversational LLMs lead to bias or privacy issues in healthcare research. Conclusions: Future studies should focus on improving the reliability of LLM applications in complex health-related tasks, as well as investigating the mechanisms of how LLM applications brought bias and privacy issues. Considering the vast accessibility of LLMs, legal, social, and technical efforts are all needed to address concerns about LLMs to promote, improve, and regularize the application of LLMs in healthcare.

Large language model application in emergency medicine and critical care

Large language models in health care: Development, applications, and challenges

Critical Care Studies Using Large Language Models Based on Electronic Healthcare Records: A Technical Note

Based on Medicine, The Now and Future of Large Language Models

The future landscape of large language models in medicine

A Systematic Review of ChatGPT and Other Conversational Large Language Models in Healthcare

Potential applications and implications of large language models in primary care

Large Language Models Like ChatGPT Show Promise, but Clinical Use of Artificial Intelligence Requires Physician Partnership to Enable Patient Care, Minimize Administrative Burden, Maximize Efficiency, and Minimize Risk

[Clinical application of large language models : Does ChatGPT replace medical report formulation? An experience report]

Transforming nursing with large language models: from concept to practice

The application of large language models in medicine: A scoping review

ChatGPT and Beyond: An overview of the growing field of large language models and their use in ophthalmology

Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review

A Survey of Clinicians’ Views of the Utility of Large Language Models

Large Language Models Illuminate a Progressive Pathway to Artificial Intelligent Healthcare Assistant

The ethics of ChatGPT in medicine and healthcare: a systematic review on Large Language Models (LLMs)

Generative Pre-Trained Transformer-Empowered Healthcare Conversations: Current Trends, Challenges, and Future Directions in Large Language Model-Enabled Medical Chatbots

A Survey of Large Language Models in Medicine: Progress, Application, and Challenge

Large language models: a primer and gastroenterology applications

Large Language Models in Medicine: The Potentials and Pitfalls

Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health