Abstract:As the health care industry increasingly embraces large language models (LLMs), understanding the consequence of this integration becomes crucial for maximizing benefits while mitigating potential pitfalls. This paper explores the evolving relationship among clinician trust in LLMs, the transition of data sources from predominantly human-generated to artificial intelligence (AI)-generated content, and the subsequent impact on the performance of LLMs and clinician competence. One of the primary concerns identified in this paper is the LLMs' self-referential learning loops, where AI-generated content feeds into the learning algorithms, threatening the diversity of the data pool, potentially entrenching biases, and reducing the efficacy of LLMs. While theoretical at this stage, this feedback loop poses a significant challenge as the integration of LLMs in health care deepens, emphasizing the need for proactive dialogue and strategic measures to ensure the safe and effective use of LLM technology. Another key takeaway from our investigation is the role of user expertise and the necessity for a discerning approach to trusting and validating LLM outputs. The paper highlights how expert users, particularly clinicians, can leverage LLMs to enhance productivity by off-loading routine tasks while maintaining a critical oversight to identify and correct potential inaccuracies in AI-generated content. This balance of trust and skepticism is vital for ensuring that LLMs augment rather than undermine the quality of patient care. We also discuss the risks associated with the deskilling of health care professionals. Frequent reliance on LLMs for critical tasks could result in a decline in health care providers' diagnostic and thinking skills, particularly affecting the training and development of future professionals. The legal and ethical considerations surrounding the deployment of LLMs in health care are also examined. We discuss the medicolegal challenges, including liability in cases of erroneous diagnoses or treatment advice generated by LLMs. The paper references recent legislative efforts, such as The Algorithmic Accountability Act of 2023, as crucial steps toward establishing a framework for the ethical and responsible use of AI-based technologies in health care. In conclusion, this paper advocates for a strategic approach to integrating LLMs into health care. By emphasizing the importance of maintaining clinician expertise, fostering critical engagement with LLM outputs, and navigating the legal and ethical landscape, we can ensure that LLMs serve as valuable tools in enhancing patient care and supporting health care professionals. This approach addresses the immediate challenges posed by integrating LLMs and sets a foundation for their maintainable and responsible use in the future.

Harnessing the Power of Large Language Models (LLMs) for Electronic Health Records (EHRs) Optimization

Critical Care Studies Using Large Language Models Based on Electronic Healthcare Records: A Technical Note

Harnessing Large Language Models in Nursing Care Planning: Opportunities, Challenges, and Ethical Considerations

Harnessing Large Language Models in Medical Research and Scientific Writing: A Closer Look to The Future

Large Language Models in Medical Education: Opportunities, Challenges, and Future Directions

Unlocking the Potential of Free Text in Electronic Health Records with Large Language Models (LLM): Enhancing Patient Safety and Consultation Interactions

Leveraging Large Language Models for Patient Engagement: The Power of Conversational AI in Digital Health

Empathy and Equity: Key Considerations for Large Language Model Adoption in Health Care

Leveraging Generative AI and Large Language Models: A Comprehensive Roadmap for Healthcare Integration

A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics

Practical Applications of Large Language Models for Health Care Professionals and Scientists

Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review

Large Language Models and User Trust: Consequence of Self-Referential Learning Loop and the Deskilling of Health Care Professionals

Embracing Large Language Models for Medical Applications: Opportunities and Challenges

Large language models in health care: Development, applications, and challenges

Large Language Model Prompting Techniques for Advancement in Clinical Medicine

Large Language Models in Medicine: The Potentials and Pitfalls

Large Language Models in Ophthalmology: Potential and Pitfalls

Large language models in healthcare and medical domain: A review

Large Language Models Illuminate a Progressive Pathway to Artificial Intelligent Healthcare Assistant