Abstract:Medical image interpretation is central to most clinical applications such as disease diagnosis, treatment planning, and prognostication. In clinical practice, radiologists examine medical images and manually compile their findings into reports, which can be a time-consuming process. Automated approaches to radiology report generation, therefore, can reduce radiologist workload and improve efficiency in the clinical pathway. While recent deep-learning approaches for automated report generation from medical images have seen some success, most studies have relied on image-derived features alone, ignoring non-imaging patient data. Although a few studies have included the word-level contexts along with the image, the use of patient demographics is still unexplored. This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information, to synthesise patient-specific radiology reports. The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information, to synthesise full-text radiology reports. Data from two public databases were used to train and evaluate the proposed approach. CXRs and reports were extracted from the MIMIC-CXR database and combined with corresponding patients' data MIMIC-IV. Based on the evaluation metrics used including patient demographic information was found to improve the quality of reports generated using the proposed approach, relative to a baseline network trained using CXRs alone. The proposed approach shows potential for enhancing radiology report generation by leveraging rich patient metadata and combining semantic text embeddings derived thereof, with medical image-derived visual features.

CT2Rep: Automated Radiology Report Generation for 3D Medical Imaging

An Inclusive Task-Aware Framework for Radiology Report Generation

Benchmarking and Boosting Radiology Report Generation for 3D High-Resolution Medical Images

CT-AGRG: Automated Abnormality-Guided Report Generation from 3D Chest CT Volumes

3D-CT-GPT: Generating 3D Radiology Reports through Integration of Large Vision-Language Models

Work like a doctor: Unifying scan localizer and dynamic generator for automated computed tomography report generation

Generating Radiology Reports via Memory-driven Transformer

Radiology Report Generation Using Transformers Conditioned with Non-imaging Data

Automatically Generating Narrative-Style Radiology Reports from Volumetric CT Images; a Proof of Concept

Medical Report Generation Via Multimodal Spatio-Temporal Fusion

Act Like a Radiologist: Radiology Report Generation across Anatomical Regions

Automated Generation of Accurate \& Fluent Medical X-ray Reports

CSAMDT: Conditional Self Attention Memory-Driven Transformers for Radiology Report Generation from Chest X-Ray

Attention based automated radiology report generation using CNN and LSTM

On the Automatic Generation of Medical Imaging Reports

HERGen: Elevating Radiology Report Generation with Longitudinal Data

Automated Radiology Report Generation: A Review of Recent Advances

Radiology Report Generation with a Learned Knowledge Base and Multi-Modal Alignment

Clinically Coherent Radiology Report Generation with Imbalanced Chest X-rays.

Radiology Report Generation via Visual Recalibration and Context Gating-Aware.

Generating radiology reports via auxiliary signal guidance and a memory-driven network