Abstract:Available large language models are not accurate enough to be deployed in real‐life clinical practice, despite their user‐friendly interfaces and rapid improvement cycles. The absence of standardised methods and benchmarks will probably delay their safe deployment into real‐life clinical settings. Summary Background Interest in large language models (LLMs), such as OpenAI's ChatGPT, across multiple specialties has grown as a source of patient‐facing medical advice and provider‐facing clinical decision support. The accuracy of LLM responses for gastroenterology and hepatology‐related questions is unknown. Aims To evaluate the accuracy and potential safety implications for LLMs for the diagnosis, management and treatment of questions related to gastroenterology and hepatology. Methods We conducted a systematic literature search including Cochrane Library, Google Scholar, Ovid Embase, Ovid MEDLINE, PubMed, Scopus and the Web of Science Core Collection to identify relevant articles published from inception until January 28, 2024, using a combination of keywords and controlled vocabulary for LLMs and gastroenterology or hepatology. Accuracy was defined as the percentage of entirely correct answers. Results Among the 1671 reports screened, we identified 33 full‐text articles on using LLMs in gastroenterology and hepatology and included 18 in the final analysis. The accuracy of question‐responding varied across different model versions. For example, accuracy ranged from 6.4% to 45.5% with ChatGPT‐3.5 and was between 40% and 91.4% with ChatGPT‐4. In addition, the absence of standardised methodology and reporting metrics for studies involving LLMs places all the studies at a high risk of bias and does not allow for the generalisation of single‐study results. Conclusions Current general‐purpose LLMs have unacceptably low accuracy on clinical gastroenterology and hepatology tasks, which may lead to adverse patient safety events through incorrect information or triage recommendations, which might overburden healthcare systems or delay necessary care.

Large language models: a primer and gastroenterology applications

Exploring the benefits and challenges of AI-driven large language models in gastroenterology: Think out of the box

The Application of Large Language Models in Gastroenterology: A Review of the Literature

Large language models in health care: Development, applications, and challenges

The future landscape of large language models in medicine

Large Language Models in Medicine: The Potentials and Pitfalls

Based on Medicine, The Now and Future of Large Language Models

A Survey of Large Language Models in Medicine: Progress, Application, and Challenge

Large language models for biomedicine: foundations, opportunities, challenges, and best practices

Large Language Models in Ophthalmology: Potential and Pitfalls

The Role of Large Language Models in Medical Education: Applications and Implications

Establishing priorities for implementation of large language models in pathology and laboratory medicine

Application Research of Large Language Models in Medicine: Status, Problems, and Future

Large language model application in emergency medicine and critical care

Systematic review: The use of large language models as medical chatbots in digestive diseases

Large Language Model Prompting Techniques for Advancement in Clinical Medicine

ChatGPT and Beyond: An overview of the growing field of large language models and their use in ophthalmology

Emerging Applications of NLP and Large Language Models in Gastroenterology and Hepatology: A Systematic Review

Large language models in radiology: fundamentals, applications, ethical considerations, risks, and future directions

A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directions