Abstract:The advent of large language models (LLMs) marks a transformative leap in natural language processing, offering unprecedented potential in radiology, particularly in enhancing the accuracy and efficiency of coronary artery disease (CAD) diagnosis. While previous studies have explored the capabilities of specific LLMs like ChatGPT in cardiac imaging, a comprehensive evaluation comparing multiple LLMs in the context of CAD-RADS 2.0 has been lacking. This study addresses this gap by assessing the performance of various LLMs, including ChatGPT 4, ChatGPT 4o, Claude 3 Opus, Gemini 1.5 Pro, Mistral Large, Meta Llama 3 70B, and Perplexity Pro, in answering 30 multiple-choice questions derived from the CAD-RADS 2.0 guidelines. Our findings reveal that ChatGPT 4o achieved the highest accuracy at 100 %, with ChatGPT 4 and Claude 3 Opus closely following at 96.6 %. Other models, including Mistral Large, Perplexity Pro, Meta Llama 3 70B, and Gemini 1.5 Pro, also demonstrated commendable performance, though with slightly lower accuracy ranging from 90 % to 93.3 %. This study underscores the proficiency of current LLMs in understanding and applying CAD-RADS 2.0, suggesting their potential to significantly enhance radiological reporting and patient care in coronary artery disease. The variations in model performance highlight the need for further research, particularly in evaluating the visual diagnostic capabilities of LLMs—a critical component of radiology practice. This study provides a foundational comparison of LLMs in CAD-RADS 2.0 and sets the stage for future investigations into their broader applications in radiology, emphasizing the importance of integrating both text-based and visual knowledge for optimal clinical outcomes.

Cosmos-LLaVA: Chatting with the Visual Cosmos-LLaVA: Görselle Sohbet Etmek

Büyük dil modellerinin Türkçe verisetleri ile eğitilmesi ve ince ayarlanması

Türkçe Dil Modellerinin Performans Karşılaştırması Performance Comparison of Turkish Language Models

Introducing cosmosGPT: Monolingual Training for Turkish Language Models

Ziya-Visual: Bilingual Large Vision-Language Model via Multi-Task Instruction Tuning

Optimizing Large Language Models for Turkish: New Methodologies in Corpus Selection and Training

VBART: The Turkish LLM

A Comprehensive Evaluation of Large Language Models for Turkish Abstractive Dialogue Summarization

Reading Gokturkish text with the Yolo object detection algorithm

LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding

Avrupa Meme Görüntüleme Diploması Çoktan Seçmeli Örnek Soruları: Büyük Dil Modellerinin Yeteneklerinin Değerlendirilmesi

MSVD-Turkish: A Comprehensive Multimodal Dataset for Integrated Vision and Language Research in Turkish

Lip Reading Using Various Deep Learning Models with Visual Turkish Data

LLaVA-VSD: Large Language-and-Vision Assistant for Visual Spatial Description

Can large language models be new supportive tools in coronary computed tomography angiography reporting?

ITACLIP: Boosting Training-Free Semantic Segmentation with Image, Text, and Architectural Enhancements

LLaVA-Endo: a Large Language-and-vision Assistant for Gastrointestinal Endoscopy

Assessing Fine-Tuning Efficacy in LLMs: A Case Study with Learning Guidance Chatbots

LLaVA-Docent: Instruction Tuning with Multimodal Large Language Model to Support Art Appreciation Education

CosmoCLIP: Generalizing Large Vision-Language Models for Astronomical Imaging

LLaVA-SpaceSGG: Visual Instruct Tuning for Open-vocabulary Scene Graph Generation with Enhanced Spatial Relations