Abstract:The integration of deep learning into radiology has the potential to enhance diagnostic processes, yet its acceptance in clinical practice remains limited due to various challenges. This study aimed to develop and evaluate a fine-tuned large language model (LLM), based on Llama 3-8B, to automate the generation of accurate and concise conclusions in magnetic resonance imaging (MRI) and computed tomography (CT) radiology reports, thereby assisting radiologists and improving reporting efficiency. A dataset comprising 15,000 radiology reports was collected from the University of Medicine and Pharmacy of Craiova's Imaging Center, covering a diverse range of MRI and CT examinations made by four experienced radiologists. The Llama 3-8B model was fine-tuned using transfer-learning techniques, incorporating parameter quantization to 4-bit precision and low-rank adaptation (LoRA) with a rank of 16 to optimize computational efficiency on consumer-grade GPUs. The model was trained over five epochs using an NVIDIA RTX 3090 GPU, with intermediary checkpoints saved for monitoring. Performance was evaluated quantitatively using Bidirectional Encoder Representations from Transformers Score (BERTScore), Recall-Oriented Understudy for Gisting Evaluation (ROUGE), Bilingual Evaluation Understudy (BLEU), and Metric for Evaluation of Translation with Explicit Ordering (METEOR) metrics on a held-out test set. Additionally, a qualitative assessment was conducted, involving 13 independent radiologists who participated in a Turing-like test and provided ratings for the AI-generated conclusions. The fine-tuned model demonstrated strong quantitative performance, achieving a BERTScore F1 of 0.8054, a ROUGE-1 F1 of 0.4998, a ROUGE-L F1 of 0.4628, and a METEOR score of 0.4282. In the human evaluation, the artificial intelligence (AI)-generated conclusions were preferred over human-written ones in approximately 21.8% of cases, indicating that the model's outputs were competitive with those of experienced radiologists. The average rating of the AI-generated conclusions was 3.65 out of 5, reflecting a generally favorable assessment. Notably, the model maintained its consistency across various types of reports and demonstrated the ability to generalize to unseen data. The fine-tuned Llama 3-8B model effectively generates accurate and coherent conclusions for MRI and CT radiology reports. By automating the conclusion-writing process, this approach can assist radiologists in reducing their workload and enhancing report consistency, potentially addressing some barriers to the adoption of deep learning in clinical practice. The positive evaluations from independent radiologists underscore the model's potential utility. While the model demonstrated strong performance, limitations such as dataset bias, limited sample diversity, a lack of clinical judgment, and the need for large computational resources require further refinement and real-world validation. Future work should explore the integration of such models into clinical workflows, address ethical and legal considerations, and extend this approach to generate complete radiology reports.

Nimg-63. Leveraging Llms For Accurate Differentiation Of Radiation Necrosis And Tumor Progression In Brain Mri Reports: A Study On Automated Scoring And Clinical Implications

Differentiating Radiation Necrosis and Metastatic Progression in Brain Tumors Using Radiomics and Machine Learning

Nimg-74. A Novel Approach For Personalized Radiotherapy Target Volume Definition Using Metabolic Mri And Deep Learning-Driven Predictions Of Glioblastoma Recurrence

NIMG-01. PREDICTING POST-STEREOTACTIC RADIOTHERAPY MAGNETIC RESONANCE IMAGE OUTCOMES OF BREAST CANCER METASTASES TO THE BRAIN

Automated Lugano Metabolic Response Assessment in 18F-Fluorodeoxyglucose–Avid Non-Hodgkin Lymphoma With Deep Learning on 18F-Fluorodeoxyglucose–Positron Emission Tomography

Abstract 6199: Defining and capturing progression in glioma by harnessing NLP in unstructured electronic health records

Leveraging Professional Radiologists' Expertise to Enhance LLMs' Evaluation for Radiology Reports

1229 Leveraging Machine Learning for Pre-operative Prediction of Supralesional Ablation in Laser Interstitial Thermal Therapy (LITT) for Brain Tumors

Cross-Institutional Structured Radiology Reporting for Lung Cancer Screening Using a Dynamic Template-Constrained Large Language Model

Development and Validation of a Dynamic-Template-Constrained Large Language Model for Generating Fully-Structured Radiology Reports

Leveraging radiomics and machine learning to differentiate radiation necrosis from recurrence in patients with brain metastases

Predictive Value of Magnetic Resonance Imaging Radiomics-Based Machine Learning for Disease Progression in Patients with High-Grade Glioma.

Natural Language Processing of Radiology Reports to Detect Complications of Ischemic Stroke

GPT-Driven Radiology Report Generation with Fine-Tuned Llama 3

Biom-54. Ai-Driven Risk-Of-Progression (Airip) Classifier For Distinguishing Recurrent Brain Metastases From Radiation Treatment Effect: A Multi-Institutional Comparative Study With Advanced Multimodal Imaging

A Deep Learning Model for Preoperative Differentiation of Glioblastoma, Brain Metastasis and Primary Central Nervous System Lymphoma: A Pilot Study

Quality Assessment of MRI-Radiomics-Based Machine Learning Methods in Classification of Brain Tumors: Systematic Review

A Multidimensional Connectomics- and Radiomics-Based Advanced Machine-Learning Framework to Distinguish Radiation Necrosis from True Progression in Brain Metastases

Nimg-88. Evaluation Of Diffusion And Perfusion Mri Features For Differentiating Recurrent Brain Metastases And Radiation Necrosis

Direct Evaluation of Treatment Response in Brain Metastatic Disease with Deep Neuroevolution