Automatic Mapping of Anatomical Landmarks from Free-Text Using Large Language Models: Insights from Llama-2

Mohamad Abdi,Gerardo Hermosillo Valadez,Halid Ziya Yerebakan

2024-10-17

Abstract:Anatomical landmarks are vital in medical imaging for navigation and anomaly detection. Modern large language models (LLMs), like Llama-2, offer promise for automating the mapping of these landmarks in free-text radiology reports to corresponding positions in image data. Recent studies propose LLMs may develop coherent representations of generative processes. Motivated by these insights, we investigated whether LLMs accurately represent the spatial positions of anatomical landmarks. Through experiments with Llama-2 models, we found that they can linearly represent anatomical landmarks in space with considerable robustness to different prompts. These results underscore the potential of LLMs to enhance the efficiency and accuracy of medical imaging workflows.

Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **How to use large - language models (LLMs), especially Llama - 2, to automatically map anatomical landmarks in free - text to their corresponding positions in image data**. Specifically, researchers hope to experimentally verify whether LLMs can accurately represent the spatial positions of anatomical landmarks, thereby improving the efficiency and accuracy of medical imaging workflows. ### Problem Background In the field of medical imaging, anatomical landmarks are crucial for navigation and anomaly detection. Radiologists usually need to interpret free - text reports and map the described anomalies to their corresponding positions in image data. This process is time - consuming and error - prone. With the development of modern large - language models (such as Llama - 2), these models perform well in interpreting and synthesizing radiology reports, so it is possible to automate this task. ### Research Motivation Although LLMs were previously considered to be only able to learn large - scale correlations and unable to understand the generation process, the latest research shows that these models may develop coherent and interpretable representations of the generation process. Based on this finding, researchers explored the following questions: - **Can the internal neural activations of LLMs accurately represent the spatial positions of anatomical landmarks?** ### Experimental Design To verify this hypothesis, researchers conducted experiments using the Llama - 2 model, predicting the positions of anatomical landmarks by linearly probing its internal neural activations. The experimental results show that: 1. The Llama - 2 model can represent anatomical landmarks in a linear space. 2. These representations are quite robust to different prompts. 3. The Llama - 2 model can also represent the sizes of anatomical landmarks in a linear space. ### Main Contributions This study demonstrates the potential application value of LLMs in medical imaging workflows, especially in automating anatomical landmark mapping. Although further research is still needed to gain a deeper understanding of these representation mechanisms and their scope, the preliminary results indicate that LLMs have the potential to significantly reduce the workload of radiologists and improve work efficiency. ### Conclusion In summary, this paper aims to explore and verify whether LLMs can accurately represent the spatial positions of anatomical landmarks, thereby providing a more efficient automated tool for medical imaging workflows.

Automatic Mapping of Anatomical Landmarks from Free-Text Using Large Language Models: Insights from Llama-2

Can large language models be new supportive tools in coronary computed tomography angiography reporting?

Advancing radiology practice and research: harnessing the potential of large language models amidst imperfections

Automated Transformation of Unstructured Cardiovascular Diagnostic Reports into Structured Datasets Using Sequentially Deployed Large Language Models

Large Language Model in Medical Informatics: Direct Classification and Enhanced Text Representations for Automatic ICD Coding

Privacy-preserving large language models for structured medical information retrieval

On Large Visual Language Models for Medical Imaging Analysis: An Empirical Study

Large Language Models: A Guide for Radiologists

Harnessing Large Language Models in Medical Research and Scientific Writing: A Closer Look to The Future

Large language models for structured reporting in radiology: past, present, and future

MGH Radiology Llama: A Llama 3 70B Model for Radiology

From Text to Tables: A Local Privacy Preserving Large Language Model for Structured Information Retrieval from Medical Documents

Revolutionizing Cardiology With Words: Unveiling the Impact of Large Language Models in Medical Science Writing

Improving Automating Quality Control in Radiology: Leveraging Large Language Models to Extract Correlative Findings in Radiology and Operative Reports

Multi-modal large language models in radiology: principles, applications, and potential

Zero-Shot LLMs for Named Entity Recognition: Targeting Cardiac Function Indicators in German Clinical Texts

Automatic structuring of radiology reports with on-premise open-source large language models

Large language models for biomedicine: foundations, opportunities, challenges, and best practices

Evaluating Large Language Models for Radiology Natural Language Processing

An efficient modular framework for automatic LIONC classification of MedIMG using unified medical language

Large language models for science and medicine