Abstract:Background: The launch of the Chat Generative Pre-trained Transformer (ChatGPT) in November 2022 has attracted public attention and academic interest to large language models (LLMs), facilitating the emergence of many other innovative LLMs. These LLMs have been applied in various fields, including healthcare. Numerous studies have since been conducted regarding how to employ state-of-the-art LLMs in health-related scenarios to assist patients, doctors, and public health administrators. Objective: This review aims to summarize the applications and concerns of applying conversational LLMs in healthcare and provide an agenda for future research on LLMs in healthcare. Methods: We utilized PubMed, ACM, and IEEE digital libraries as primary sources for this review. We followed the guidance of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRIMSA) to screen and select peer-reviewed research articles that (1) were related to both healthcare applications and conversational LLMs and (2) were published before September 1st, 2023, the date when we started paper collection and screening. We investigated these papers and classified them according to their applications and concerns. Results: Our search initially identified 820 papers according to targeted keywords, out of which 65 papers met our criteria and were included in the review. The most popular conversational LLM was ChatGPT from OpenAI (60), followed by Bard from Google (1), Large Language Model Meta AI (LLaMA) from Meta (1), and other LLMs (5). These papers were classified into four categories in terms of their applications: 1) summarization, 2) medical knowledge inquiry, 3) prediction, and 4) administration, and four categories of concerns: 1) reliability, 2) bias, 3) privacy, and 4) public acceptability. There are 49 (75%) research papers using LLMs for summarization and/or medical knowledge inquiry, and 58 (89%) research papers expressing concerns about reliability and/or bias. We found that conversational LLMs exhibit promising results in summarization and providing medical knowledge to patients with a relatively high accuracy. However, conversational LLMs like ChatGPT are not able to provide reliable answers to complex health-related tasks that require specialized domain expertise. Additionally, no experiments in our reviewed papers have been conducted to thoughtfully examine how conversational LLMs lead to bias or privacy issues in healthcare research. Conclusions: Future studies should focus on improving the reliability of LLM applications in complex health-related tasks, as well as investigating the mechanisms of how LLM applications brought bias and privacy issues. Considering the vast accessibility of LLMs, legal, social, and technical efforts are all needed to address concerns about LLMs to promote, improve, and regularize the application of LLMs in healthcare.

Generative Pre-Trained Transformer-Empowered Healthcare Conversations: Current Trends, Challenges, and Future Directions in Large Language Model-Enabled Medical Chatbots

Leveraging Large Language Models for Patient Engagement: The Power of Conversational AI in Digital Health

Large language model application in emergency medicine and critical care

A Systematic Review of ChatGPT and Other Conversational Large Language Models in Healthcare

Leveraging Generative AI and Large Language Models: A Comprehensive Roadmap for Healthcare Integration

Large language models in health care: Development, applications, and challenges

Based on Medicine, The Now and Future of Large Language Models

Transformative Trends: A Comprehensive Review of Large Language Models (LLMs) in Healthcare

The Breakthrough of Large Language Models Release for Medical Applications: 1-Year Timeline and Perspectives

Ethical Considerations of Artificial Intelligence in Health Care: Examining the Role of Generative Pretrained Transformer-4

Critical Care Studies Using Large Language Models Based on Electronic Healthcare Records: A Technical Note

Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review

Potential applications and implications of large language models in primary care

Healthcare: A Growing Role for Large Language Models and Generative AI

Utilizing Large Language Models in Ophthalmology: The Current Landscape and Challenges

Large Language Models Illuminate a Progressive Pathway to Artificial Intelligent Healthcare Assistant

AI as a Medical Ally: Evaluating ChatGPT's Usage and Impact in Indian Healthcare

Understanding and training for the impact of large language models and artificial intelligence in healthcare practice: a narrative review

A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine

The Impact of Multimodal Large Language Models on Health Care's Future