Abstract:Electronic health record adoption has produced tremendous growth in patient-physician inbox messages, 1 which accelerated during the COVID-19 pandemic, 2 ,3 with some health systems instituting charges for this mounting clinical burden. 4 ,5 However, message content across populations is poorly understood, limiting systematic approaches that can improve message-related workloads. The Kaiser Permanente Northern California (KPNC) Research Determination Office deemed this quality improvement study to be exempt from review and did not require informed consent because it was not human participants research. We followed the STROBE reporting guideline. In 2019, KPNC initiated the Desktop Medicine Program to operationalize a solution to address messaging volume and expeditiously direct patients to the optimal person, information, or service (eAppendix in Supplement 1). Program development integrated regional clinician teams and real-time natural language processing (NLP) algorithms to label message content and sort messages to appropriate roles (eg, teleservice representatives, medical assistants, pharmacists, physicians) before reaching individual physician inboxes. Category labels were added to the NLP output when specific Desktop Medicine response workflows were in place. The NLP pipeline was trained on a corpus of more than 20 000 messages annotated by nurses with triage experience (eAppendix in Supplement 1) and integrated within the electronic health record in May 2022 with bimonthly programmatic updates. The program initially focused on adult and family medicine messages and subsequently expanded to pediatric messages in October 2022. We examined the content and variability in message labels for adult and family medicine and pediatric physicians between April 1 and August 31, 2023, with descriptive statistics and coefficient of variation and quantified the association with physician inbox volume. Statistical analysis was performed using Python version 3.10.8 (Python Software Foundation), Pandas package version 1.5.3, and Microsoft Excel. There were 4 709 261 patient messages during the study period, of which 3 655 065 (77.6%]) received at least 1 label, most commonly for medications (1 546 411 [32.8%]), skin conditions (438 631 [9.3%]), messages with attachments (389 372 [8.3%]), and emergent content for expedited review (359 144 [7.6%]) (Table). Overall, 1 374 309 messages (29.2%) included at least 2 labels, most commonly including skin conditions with medications (179 707 [4.1%]), skin conditions with attachments (171 955 [4.0%]), and emergent content with medications (148 357 [3.4%]). Weekly label variability (Figure) was lowest for controlled substances (coefficient of variation: 4.2 [95% CI, 3.2-6.1]), medication (4.5 [95% CI, 3.4-6.5]), neck and back pain (4.7 [95% CI, 3.6-6.8]), and erectile dysfunction (5.3 [95% CI, 4.1-7.6]) and highest for influenza vaccine (96.4 [95% CI, 73.8-139.3]), Mpox (43.1 [95% CI, 33.0-62.2]), and COVID-19 vaccine (35.2 [95% CI, 26.9-50.8]). Of 4 835 200 messages evaluated by regional Desktop Medicine staff, 1 541 657 (31.9%) were resolved by regional medical assistants or teleservice representatives, physicians, and pharmacists before reaching individual physician inboxes. This study including more than 4 million patient messages found novel insights that inform health system approaches for addressing the high volume of patient-physician electronic communications in a timely and effective fashion. First, identifying common and recurring categories (eg, medication-related or skin condition–related labels) across many physicians informs the development of population-level approaches for reducing individual physician inbox message volume. Second, quantifying the variability in message categories highlights acute changes in specific categories that can be used to identify emerging trends and facilitate rapid operational responses (eg, COVID-19, influenza, or Mpox). Third, real-time message analysis, labeling content before reaching a physician inbox for expedited regional review, can shorten the time to clinical assessment for potentially emergent conditions. Additionally, although most messages were felt to be most appropriately addressed by the intended physician recipient, the program was effective at resolving more than 1.5 million patient messages, accounting for nearly one-third of overall volume. Study limitations included that the findings were from a single health care system and that the program did not include newer large language models (eg, GPT-4), 6 which could improve content labeling and responses. Future enhancements are also needed to effectively address messages with multiple topics and those with no category labels. In conclusion, this study's results suggest that a health system–wide approach to classifying patient messages paired with well-defined -Abstract Truncated-

What problem does this paper attempt to address?

The paper aims to address the workload issues caused by the extensive information exchange between patients and doctors in electronic health record systems. Specifically, the researchers analyzed a large number of patient electronic messages to identify common message categories and used natural language processing (NLP) techniques to automatically classify and triage these messages, thereby reducing doctors' workload and improving information processing efficiency. The main objectives of the study include: 1. **Identifying common message categories**: By analyzing over 4 million patient messages, the researchers identified the most common message categories, such as medication-related and skin conditions. 2. **Quantifying changes in message categories**: Using statistical methods, the researchers quantified the changes in different message categories to promptly identify and respond to emerging trends. 3. **Optimizing information processing workflows**: By utilizing real-time NLP algorithms to classify and triage messages before they reach doctors' inboxes, the response time to potential emergencies is shortened. 4. **Reducing doctors' workload**: By redirecting some messages directly to other healthcare staff (such as phone service representatives, pharmacists, etc.), the information processing burden on doctors is alleviated. The study results indicate that this approach can effectively reduce the information processing pressure on doctors and improve the overall response efficiency of the healthcare system.

Content of Patient Electronic Messages to Physicians in a Large Integrated System

Understanding physicians' work via text analytics on EHR inbox messages

Are AI-Generated Replies the Answer to the EHR Inbox Problem?

Demographics and clinical features associated with rates of electronic message utilization in the primary care setting

Development and evaluation of an artificial intelligence-based workflow for the prioritization of patient portal messages

Perspectives on Artificial Intelligence–Generated Responses to Patient Messages

Look Who's Talking: Application of a Theory-Based Taxonomy to Patient-Clinician E-mail Messages

Patient Use of Email, Facebook, and Physician Websites to Communicate with Physicians: A National Online Survey of Retail Pharmacy Users

Identifying Medication-Related Intents From a Bidirectional Text Messaging Platform for Hypertension Management Using an Unsupervised Learning Approach: Retrospective Observational Pilot Study

Practicing Clinicians' Recommendations to Reduce Burden from the Electronic Health Record Inbox: a Mixed-Methods Study

We Ask and Listen: A Group-Wide Retrospective Survey on Satisfaction with Digital Medication Software

Large Language Model-Based Responses to Patients' In-Basket Messages

Prompt engineering on leveraging large language models in generating response to InBasket messages

Prompt matters: evaluation of large language model chatbot responses related to Peyronie's disease

Distractions During Patient Handoff: The Application-Based Messaging Volume on General Surgery Interns

Implementation of Patient Engagement Tools in Electronic Health Records to Enhance Patient-Centered Communication: Protocol for Feasibility Evaluation and Preliminary Results

Prompt matters: evaluation of large language model chatbot responses related to Peyronie’s disease

Leveraging large language models for generating responses to patient messages-a subjective analysis

Quantifying the impact of telemedicine and patient medical advice request messages on physicians' work-outside-work

Factors influencing physician responsiveness to nurse-initiated communication: a qualitative study

Patient-initiated Electronic Messages and Quality of Care for Patients With Diabetes and Hypertension in a Large Fee-for-Service Medical Group: Results From a Natural Experiment