Abstract:Brain diseases exert profound detrimental effects on human health by affecting the central nervous system. Accurate automated diagnosis of brain diseases is imperative to delay the progression of illness and enhance long-term prognosis. However, existing image-based diagnostic approaches struggle to achieve satisfactory performance due to the high dimensionality of imaging data. Radiological reports, which are required in clinical routine to describe image findings, provide a more straightforward comprehension of the imaging data, yet they have been neglected in automated brain disease classification. In this work, we explore automated brain disease classification via radiological reports and language models and compare the results with conventional image-based methods. Specifically, in the report-based diagnostic approach, we fine-tune Pre-trained Language Models (PLMs) and Large Language Models (LLMs) based on the findings part of radiological reports to achieve disease classification. Four clinically relevant brain disease classification tasks were performed in our experiments, involving 12 datasets with a total number of 14,970 patients, including two independent validation sets. The best language model reached an average area under the receiver operating characteristic curve (AUC) of 84.75%, an average accuracy (ACC) of 79.48%, and an average F1-score of 79.45%. Compared with the best image-based model, it achieved an average improvement of 10.34%, 10.75%, and 9.95% in terms of AUC, ACC, and F1-score, respectively. The language model also outperformed junior radiologists by 9.47% in terms of ACC. Moreover, the report-based model exhibited better adaptability to missing image contrasts and cross-site data variability than image-based models. Together, these results show that brain disease classification via language model analysis of radiological reports can be more reliable than image-based classification, and our work demonstrates the potential of using radiological reports for accurate diagnosis of brain diseases.

Prior tissue knowledge-driven contrastive learning for brain CT report generation

A Novel Method Of Synthetic Ct Generation From Mr Images Based On Convolutional Neural Networks

MKCL: Medical Knowledge with Contrastive Learning model for radiology report generation

GHCL: Gaussian heuristic curriculum learning for Brain CT report generation

Granularity Matters: Pathological Graph-driven Cross-modal Alignment for Brain CT Report Generation

Visual prior-based cross-modal alignment network for radiology report generation

Weakly Guided Attention Model with Hierarchical Interaction for Brain CT Report Generation.

Medical-VLBERT: Medical Visual Language BERT for COVID-19 CT Report Generation With Alternate Learning

Cross-modal Contrastive Attention Model for Medical Report Generation.

See Detail Say Clear: Towards Brain CT Report Generation via Pathological Clue-driven Representation Learning

Interactive dual-stream contrastive learning for radiology report generation

Multi-Grained Radiology Report Generation With Sentence-Level Image-Language Contrastive Learning

Simple Words over Rich Imaging: Accurate Brain Disease Classification via Language Model Analysis of Radiological Reports

Large Language Model with Region-guided Referring and Grounding for CT Report Generation

Work like a doctor: Unifying scan localizer and dynamic generator for automated computed tomography report generation

Towards a Holistic Framework for Multimodal Large Language Models in Three-dimensional Brain CT Report Generation

Medical Report Generation based on Segment-Enhanced Contrastive Representation Learning

Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation

Radiology Report Generation via Structured Knowledge-Enhanced Multi-modal Attention and Contrastive Learning.

Boosting Radiology Report Generation by Infusing Comparison Prior

Dia-LLaMA: Towards Large Language Model-driven CT Report Generation