MAGDA: Multi-agent guideline-driven diagnostic assistance

David Bani-Harouni,Nassir Navab,Matthias Keicher

2024-09-10

Abstract:In emergency departments, rural hospitals, or clinics in less developed regions, clinicians often lack fast image analysis by trained radiologists, which can have a detrimental effect on patients' healthcare. Large Language Models (LLMs) have the potential to alleviate some pressure from these clinicians by providing insights that can help them in their decision-making. While these LLMs achieve high test results on medical exams showcasing their great theoretical medical knowledge, they tend not to follow medical guidelines. In this work, we introduce a new approach for zero-shot guideline-driven decision support. We model a system of multiple LLM agents augmented with a contrastive vision-language model that collaborate to reach a patient diagnosis. After providing the agents with simple diagnostic guidelines, they will synthesize prompts and screen the image for findings following these guidelines. Finally, they provide understandable chain-of-thought reasoning for their diagnosis, which is then self-refined to consider inter-dependencies between diseases. As our method is zero-shot, it is adaptable to settings with rare diseases, where training data is limited, but expert-crafted disease descriptions are available. We evaluate our method on two chest X-ray datasets, CheXpert and ChestX-ray 14 Longtail, showcasing performance improvement over existing zero-shot methods and generalizability to rare diseases.

Artificial Intelligence

What problem does this paper attempt to address?

The paper attempts to address the challenges faced by clinicians in performing rapid image analysis in emergency departments, rural hospitals, or clinics in underdeveloped areas due to the lack of trained radiologists. This not only affects the quality of patient healthcare but also increases the workload of clinicians.

Specifically, the paper proposes a novel multi-agent system (MAGDA) that leverages large language models (LLMs) and contrastive vision-language models to collaboratively diagnose patients. This system can generate prompts based on simple diagnostic guidelines and screen images for abnormalities, ultimately providing understandable chain-of-thought reasoning to support clinical decision-making. This approach is particularly suitable for diagnosing rare diseases, as data for these diseases are limited, but expert-written disease descriptions are available.

The main contributions of the paper include:
1. Proposing an end-to-end guideline-based approach that requires only clinical guidelines and medical images as input for zero-shot diagnosis.
2. Introducing dynamic prompting techniques that enable LLMs to screen for unknown diseases in medical images without fine-tuning.
3. Providing a transparent diagnostic process through chain-of-thought reasoning, enhancing the interpretability of diagnostic results.