Abstract:Electronic health record adoption has produced tremendous growth in patient-physician inbox messages, 1 which accelerated during the COVID-19 pandemic, 2 ,3 with some health systems instituting charges for this mounting clinical burden. 4 ,5 However, message content across populations is poorly understood, limiting systematic approaches that can improve message-related workloads. The Kaiser Permanente Northern California (KPNC) Research Determination Office deemed this quality improvement study to be exempt from review and did not require informed consent because it was not human participants research. We followed the STROBE reporting guideline. In 2019, KPNC initiated the Desktop Medicine Program to operationalize a solution to address messaging volume and expeditiously direct patients to the optimal person, information, or service (eAppendix in Supplement 1). Program development integrated regional clinician teams and real-time natural language processing (NLP) algorithms to label message content and sort messages to appropriate roles (eg, teleservice representatives, medical assistants, pharmacists, physicians) before reaching individual physician inboxes. Category labels were added to the NLP output when specific Desktop Medicine response workflows were in place. The NLP pipeline was trained on a corpus of more than 20 000 messages annotated by nurses with triage experience (eAppendix in Supplement 1) and integrated within the electronic health record in May 2022 with bimonthly programmatic updates. The program initially focused on adult and family medicine messages and subsequently expanded to pediatric messages in October 2022. We examined the content and variability in message labels for adult and family medicine and pediatric physicians between April 1 and August 31, 2023, with descriptive statistics and coefficient of variation and quantified the association with physician inbox volume. Statistical analysis was performed using Python version 3.10.8 (Python Software Foundation), Pandas package version 1.5.3, and Microsoft Excel. There were 4 709 261 patient messages during the study period, of which 3 655 065 (77.6%]) received at least 1 label, most commonly for medications (1 546 411 [32.8%]), skin conditions (438 631 [9.3%]), messages with attachments (389 372 [8.3%]), and emergent content for expedited review (359 144 [7.6%]) (Table). Overall, 1 374 309 messages (29.2%) included at least 2 labels, most commonly including skin conditions with medications (179 707 [4.1%]), skin conditions with attachments (171 955 [4.0%]), and emergent content with medications (148 357 [3.4%]). Weekly label variability (Figure) was lowest for controlled substances (coefficient of variation: 4.2 [95% CI, 3.2-6.1]), medication (4.5 [95% CI, 3.4-6.5]), neck and back pain (4.7 [95% CI, 3.6-6.8]), and erectile dysfunction (5.3 [95% CI, 4.1-7.6]) and highest for influenza vaccine (96.4 [95% CI, 73.8-139.3]), Mpox (43.1 [95% CI, 33.0-62.2]), and COVID-19 vaccine (35.2 [95% CI, 26.9-50.8]). Of 4 835 200 messages evaluated by regional Desktop Medicine staff, 1 541 657 (31.9%) were resolved by regional medical assistants or teleservice representatives, physicians, and pharmacists before reaching individual physician inboxes. This study including more than 4 million patient messages found novel insights that inform health system approaches for addressing the high volume of patient-physician electronic communications in a timely and effective fashion. First, identifying common and recurring categories (eg, medication-related or skin condition–related labels) across many physicians informs the development of population-level approaches for reducing individual physician inbox message volume. Second, quantifying the variability in message categories highlights acute changes in specific categories that can be used to identify emerging trends and facilitate rapid operational responses (eg, COVID-19, influenza, or Mpox). Third, real-time message analysis, labeling content before reaching a physician inbox for expedited regional review, can shorten the time to clinical assessment for potentially emergent conditions. Additionally, although most messages were felt to be most appropriately addressed by the intended physician recipient, the program was effective at resolving more than 1.5 million patient messages, accounting for nearly one-third of overall volume. Study limitations included that the findings were from a single health care system and that the program did not include newer large language models (eg, GPT-4), 6 which could improve content labeling and responses. Future enhancements are also needed to effectively address messages with multiple topics and those with no category labels. In conclusion, this study's results suggest that a health system–wide approach to classifying patient messages paired with well-defined -Abstract Truncated-

Safety risks and workflow implications associated with nursing-related free-text communication orders

Physician-Nurse Communication Surrounding Computerized Physician Order Entry Systems From Social and Technical Perspective: An Ethnographic Study

Nursing risks after implementation of an electronic prescription system in department of hematology:aqualitative research

Facilitated Nurse Medication-Related Event Reporting to Improve Medication Management Quality and Safety in Intensive Care Units

Secure Messaging Use and Wrong-Patient Errors

Electronic health record system use and documentation burden of acute and critical care nurse clinicians: a mixed-methods study

Examining nursing vital signs documentation workflow: barriers and opportunities in general internal medicine units

Secure Messaging Use and Wrong-Patient Ordering Errors by Inpatient Clinicians

Medication Safety Gaps in English Pediatric Inpatient Units: An Exploration Using Work Domain Analysis

Electronic distractions of the respiratory therapist and their impact on patient safety

Implementing Electronic Discharge Communication Tools in Pediatric Emergency Departments: Multicountry, Cross-Sectional Readiness Survey of Nurses and Physicians

Academic hospitalist perspectives on the benefits and challenges of secure messaging: A mixed methods analysis

Distractions During Patient Handoff: The Application-Based Messaging Volume on General Surgery Interns

e-Prescribing: characterisation of patient safety hazards in community pharmacies using a sociotechnical systems approach

[Miscommunication as a risk focus in patient safety : Work process analysis in prehospital emergency care]

Nursing implications of an early warning system implemented to reduce adverse events: a qualitative study

Frequency and Types of Patient-Reported Errors in Electronic Health Record Ambulatory Care Notes

Content of Patient Electronic Messages to Physicians in a Large Integrated System

Nurses' role in medication safety

Technology Based Challenges of Informal Clinical Communication in an Australian tertiary referral hospital : A mixed methods assessment of The Need for Change.

Visual analysis of patient safety research hotspots and trends in the context of telemedicine based on Web of Science