Abstract:Objective: We present a proof-of-concept digital scribe system as an Emergency Department (ED) consultation call-based clinical conversation summarization pipeline to support clinical documentation, and report its performance. Materials and methods: We use four pre-trained large language models to establish the digital scribe system: T5-small, T5-base, PEGASUS-PubMed, and BART-Large-CNN via zero-shot and fine-tuning approaches. Our dataset includes 100 referral conversations among ED clinicians and medical records. We report the ROUGE-1, ROUGE-2, and ROUGE-L to compare model performance. In addition, we annotated transcriptions to assess the quality of generated summaries. Results: The fine-tuned BART-Large-CNN model demonstrates greater performance in summarization tasks with the highest ROUGE scores (F1ROUGE-1=0.49, F1ROUGE-2=0.23, F1ROUGE-L=0.35) scores. In contrast, PEGASUS-PubMed lags notably (F1ROUGE-1=0.28, F1ROUGE-2=0.11, F1ROUGE-L=0.22). BART-Large-CNN's performance decreases by more than 50% with the zero-shot approach. Annotations show that BART-Large-CNN performs 71.4% recall in identifying key information and a 67.7% accuracy rate. Discussion: The BART-Large-CNN model demonstrates a high level of understanding of clinical dialogue structure, indicated by its performance with and without fine-tuning. Despite some instances of high recall, there is variability in the model's performance, particularly in achieving consistent correctness, suggesting room for refinement. The model's recall ability varies across different information categories. Conclusion: The study provides evidence towards the potential of AI-assisted tools in assisting clinical documentation. Future work is suggested on expanding the research scope with additional language models and hybrid approaches, and comparative analysis to measure documentation burden and human factors.

Automatic analysis of medical dialogue in the home hemodialysis domain: Structure induction and summarization

Generating medically-accurate summaries of patient-provider dialogue: A multi-stage approach using large language models

Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structures

Towards an Automated SOAP Note: Classifying Utterances from Medical Conversations

A Factual Aware Two-Stage Model for Medical Dialogue Summarization.

Automated Lay Language Summarization of Biomedical Scientific Reviews

Towards Efficient Medical Dialogue Summarization with Compacting-Abstractive Model.

Evaluation of a Digital Scribe: Conversation Summarization for Emergency Department Consultation Calls

Generate Descriptions of Medical Dialogues Through Two-Layers Transformer-Based Summarization

MedicalSum: A Guided Clinical Abstractive Summarization Model for Generating Medical Reports from Patient-Doctor Conversations

Summarizing Medical Conversations via Identifying Important Utterances

Development and Evaluation of a Digital Scribe: Conversation Summarization Pipeline for Emergency Department Counseling Sessions towards Reducing Documentation Burden

MEDSAGE: Enhancing Robustness of Medical Dialogue Summarization to ASR Errors with LLM-generated Synthetic Dialogues

CLINICSUM: Utilizing Language Models for Generating Clinical Summaries from Patient-Doctor Conversations

Real-time Speech Summarization for Medical Conversations

COSSUM: Towards Conversation-Oriented Structured Summarization for Automatic Medical Insurance Assessment

Enhancing Summarization Performance through Transformer-Based Prompt Engineering in Automated Medical Reporting

Hierarchical Summarization for Longform Spoken Dialog

Two eyes, Two views, and finally, One summary! Towards Multi-modal Multi-tasking Knowledge-Infused Medical Dialogue Summarization

MVP: Optimizing Multi-view Prompts for Medical Dialogue Summarization.

Synthetic Patient-Physician Dialogue Generation from Clinical Notes Using LLM