Abstract:Automatic diagnosis is a significant application of AI in healthcare, where diagnoses are generated based on the symptom description of patients. Previous works have approached this task directly by modeling the relationship between the normalized symptoms and all possible diseases. However, in the clinical diagnostic process, patients are initially consulted by a general practitioner and, if necessary, referred to specialists in specific domains for a more comprehensive evaluation. The final diagnosis often emerges from a collaborative consultation among medical specialist groups. Recently, large language models have shown impressive capabilities in natural language understanding. In this study, we adopt tuning-free LLM-based agents as medical practitioners and propose the Agent-derived Multi-Specialist Consultation (AMSC) framework to model the diagnosis process in the real world by adaptively fusing probability distributions of agents over potential diseases. Experimental results demonstrate the superiority of our approach compared with baselines. Notably, our approach requires significantly less parameter updating and training time, enhancing efficiency and practical utility. Furthermore, we delve into a novel perspective on the role of implicit symptoms within the context of automatic diagnosis.

What problem does this paper attempt to address?

The paper aims to address two main issues in the field of automated diagnosis: 1. **Improving automated diagnostic systems to simulate real-world clinical diagnostic processes**: Existing automated diagnostic methods typically model the relationship between symptoms and diseases directly, neglecting the general process in actual clinical settings—where patients are first seen by a general practitioner and then referred to a specialist for further evaluation if necessary. The paper proposes a method based on large language models (LLM) to simulate this multi-specialist consultation process. 2. **Enhancing the efficiency and practicality of automated diagnostic systems**: By developing an Agent-derived Multi-Specialist Consultation (AMSC) framework that does not require updating LLM parameters, the framework can effectively integrate the probability distributions of potential diseases from different expert agents, thereby improving diagnostic accuracy while reducing training time and the need for parameter updates. Specifically, the main contributions of the paper include: - Proposing an LLM-based multi-specialist consultation model that applies adaptive predictive distribution fusion methods for disease diagnosis in real-world clinical scenarios. - Experimental results show that this method has significant performance advantages over baseline methods and is more efficient in terms of training costs. - Conducting a comprehensive analysis of the impact of symptomatology on automated diagnosis, exploring the different effects of various types of symptoms on diagnostic outcomes. To achieve these goals, the paper details the construction process of the AMSC framework, including how to refine the automated diagnostic tasks, implement agent-based expert diagnosis, and multi-expert decision fusion techniques. Additionally, the paper validates the effectiveness and efficiency of the AMSC framework through experiments and discusses its performance in handling explicit and implicit symptoms in depth.

Beyond Direct Diagnosis: LLM-based Multi-Specialist Agent Consultation for Automatic Diagnosis

AI Hospital: Interactive Evaluation and Collaboration of LLMs As Intern Doctors for Clinical Diagnosis

Enhancing Diagnostic Accuracy through Multi-Agent Conversations: Using Large Language Models to Mitigate Cognitive Bias

An Automatic Evaluation Framework for Multi-turn Medical Consultations Capabilities of Large Language Models

AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator

KG4Diagnosis: A Hierarchical Multi-Agent LLM Framework with Knowledge Graph Enhancement for Medical Diagnosis

Towards Automatic Evaluation for LLMs' Clinical Capabilities: Metric, Data, and Algorithm

Adaptive Reasoning and Acting in Medical Language Agents

MTDiag: An Effective Multi-Task Framework for Automatic Diagnosis

RareAgents: Autonomous Multi-disciplinary Team for Rare Disease Diagnosis and Treatment

MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning

Conversational Disease Diagnosis via External Planner-Controlled Large Language Models

MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making

Automatic Interactive Evaluation for Large Language Models with State Aware Patient Simulator

MAGDA: Multi-agent guideline-driven diagnostic assistance

MedAide: Towards an Omni Medical Aide via Specialized LLM-based Multi-Agent Collaboration

Evaluating large language models as agents in the clinic

Large Language Models as Agents in the Clinic

Towards Conversational Diagnostic AI

Large Language Models Illuminate a Progressive Pathway to Artificial Intelligent Healthcare Assistant