Abstract:As the health care industry increasingly embraces large language models (LLMs), understanding the consequence of this integration becomes crucial for maximizing benefits while mitigating potential pitfalls. This paper explores the evolving relationship among clinician trust in LLMs, the transition of data sources from predominantly human-generated to artificial intelligence (AI)-generated content, and the subsequent impact on the performance of LLMs and clinician competence. One of the primary concerns identified in this paper is the LLMs' self-referential learning loops, where AI-generated content feeds into the learning algorithms, threatening the diversity of the data pool, potentially entrenching biases, and reducing the efficacy of LLMs. While theoretical at this stage, this feedback loop poses a significant challenge as the integration of LLMs in health care deepens, emphasizing the need for proactive dialogue and strategic measures to ensure the safe and effective use of LLM technology. Another key takeaway from our investigation is the role of user expertise and the necessity for a discerning approach to trusting and validating LLM outputs. The paper highlights how expert users, particularly clinicians, can leverage LLMs to enhance productivity by off-loading routine tasks while maintaining a critical oversight to identify and correct potential inaccuracies in AI-generated content. This balance of trust and skepticism is vital for ensuring that LLMs augment rather than undermine the quality of patient care. We also discuss the risks associated with the deskilling of health care professionals. Frequent reliance on LLMs for critical tasks could result in a decline in health care providers' diagnostic and thinking skills, particularly affecting the training and development of future professionals. The legal and ethical considerations surrounding the deployment of LLMs in health care are also examined. We discuss the medicolegal challenges, including liability in cases of erroneous diagnoses or treatment advice generated by LLMs. The paper references recent legislative efforts, such as The Algorithmic Accountability Act of 2023, as crucial steps toward establishing a framework for the ethical and responsible use of AI-based technologies in health care. In conclusion, this paper advocates for a strategic approach to integrating LLMs into health care. By emphasizing the importance of maintaining clinician expertise, fostering critical engagement with LLM outputs, and navigating the legal and ethical landscape, we can ensure that LLMs serve as valuable tools in enhancing patient care and supporting health care professionals. This approach addresses the immediate challenges posed by integrating LLMs and sets a foundation for their maintainable and responsible use in the future.

Large language models in physical therapy: time to adapt and adept

Large Language Models in Medical Education: Opportunities, Challenges, and Future Directions

Understanding and training for the impact of large language models and artificial intelligence in healthcare practice: a narrative review

Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review

Large Language Models in Ophthalmology: Potential and Pitfalls

Based on Medicine, The Now and Future of Large Language Models

Large Language Models Illuminate a Progressive Pathway to Artificial Intelligent Healthcare Assistant

Large Language Models and User Trust: Consequence of Self-Referential Learning Loop and the Deskilling of Health Care Professionals

Practical Applications of Large Language Models for Health Care Professionals and Scientists

Large Language Model Prompting Techniques for Advancement in Clinical Medicine

Potential applications and implications of large language models in primary care

Large language models could change the future of behavioral healthcare: a proposal for responsible development and evaluation

The future landscape of large language models in medicine

Establishing priorities for implementation of large language models in pathology and laboratory medicine

Large language models in health care: Development, applications, and challenges

Large language model application in emergency medicine and critical care

Transforming nursing with large language models: from concept to practice

AI as a Medical Ally: Evaluating ChatGPT's Usage and Impact in Indian Healthcare

Harnessing Large Language Models in Nursing Care Planning: Opportunities, Challenges, and Ethical Considerations

Large Language Models in Medicine: The Potentials and Pitfalls

Harnessing the potential of large language models in medical education: promise and pitfalls