Abstract:Objectives: Provide physicians and researchers an efficient way to extract information from weakly structured radiology reports with natural language processing (NLP) machine learning models. Methods: We evaluate seven different German bidirectional encoder representations from transformers (BERT) models on a dataset of 857,783 unlabeled radiology reports and an annotated reading comprehension dataset in the format of SQuAD 2.0 based on 1223 additional reports. Results: Continued pre-training of a BERT model on the radiology dataset and a medical online encyclopedia resulted in the most accurate model with an F1-score of 83.97% and an exact match score of 71.63% for answerable questions and 96.01% accuracy in detecting unanswerable questions. Fine-tuning a non-medical model without further pre-training led to the lowest-performing model. The final model proved stable against variation in the formulations of questions and in dealing with questions on topics excluded from the training set. Conclusions: General domain BERT models further pre-trained on radiological data achieve high accuracy in answering questions on radiology reports. We propose to integrate our approach into the workflow of medical practitioners and researchers to extract information from radiology reports. Clinical relevance statement: By reducing the need for manual searches of radiology reports, radiologists' resources are freed up, which indirectly benefits patients. Key points: • BERT models pre-trained on general domain datasets and radiology reports achieve high accuracy (83.97% F1-score) on question-answering for radiology reports. • The best performing model achieves an F1-score of 83.97% for answerable questions and 96.01% accuracy for questions without an answer. • Additional radiology-specific pretraining of all investigated BERT models improves their performance.

ChestXRayBERT: A Pretrained Language Model for Chest Radiology Report Summarization

Learning to Summarize Chinese Radiology Findings With a Pre-Trained Encoder

An Inclusive Task-Aware Framework for Radiology Report Generation

RadBARTsum: Domain Specific Adaption of Denoising Sequence-to-Sequence Models for Abstractive Radiology Report Summarization

Multilingual Natural Language Processing Model for Radiology Reports -- The Summary is all you need!

Improving Expert Radiology Report Summarization by Prompting Large Language Models with a Layperson Summary

Summarizing Radiology Reports Findings into Impressions

Patient Centric Summarization of Radiology Findings using Large Language Models

Leveraging Summary Guidance on Medical Report Summarization

An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT

XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models

Clinically Accurate Chest X-Ray Report Generation

Biomedical-domain Pre-Trained Language Model for Extractive Summarization

Understanding transfer learning for chest radiograph clinical report generation with modified transformer architectures

Expert Insight-Enhanced Follow-up Chest X-Ray Summary Generation

CSAMDT: Conditional Self Attention Memory-Driven Transformers for Radiology Report Generation from Chest X-Ray

Highly accurate classification of chest radiographic reports using a deep learning natural language model pre-trained on 3.8 million text reports

Beyond Images: An Integrative Multi-modal Approach to Chest X-Ray Report Generation

Information extraction from weakly structured radiological reports with natural language queries

Prompt-Guided Generation of Structured Chest X-Ray Report Using a Pre-trained LLM

Evaluation of large language models performance against humans for summarizing MRI knee radiology reports: A feasibility study