Abstract:Recent advances in Large Language Models (LLMs) have achieved remarkable breakthroughs in understanding and responding to user intents. However, their performance lag behind general use cases in some expertise domains, such as Chinese medicine. Existing efforts to incorporate Chinese medicine into LLMs rely on Supervised Fine-Tuning (SFT) with single-turn and distilled dialogue data. These models lack the ability for doctor-like proactive inquiry and multi-turn comprehension and cannot align responses with experts' intentions. In this work, we introduce Zhongjing, the first Chinese medical LLaMA-based LLM that implements an entire training pipeline from continuous pre-training, SFT, to Reinforcement Learning from Human Feedback (RLHF). Additionally, we construct a Chinese multi-turn medical dialogue dataset of 70,000 authentic doctor-patient dialogues, CMtMedQA, which significantly enhances the model's capability for complex dialogue and proactive inquiry initiation. We also define a refined annotation rule and evaluation criteria given the unique characteristics of the biomedical domain. Extensive experimental results show that Zhongjing outperforms baselines in various capacities and matches the performance of ChatGPT in some abilities, despite the 100x parameters. Ablation studies also demonstrate the contributions of each component: pre-training enhances medical knowledge, and RLHF further improves instruction-following ability and safety. Our code, datasets, and models are available at <a class="link-external link-https" href="https://github.com/SupritYoung/Zhongjing" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve The paper aims to address the application of large language models (LLMs) in the field of Traditional Chinese Medicine (TCM). Although existing large language models (such as ChatGPT and GPT-4) perform well on many general tasks, their performance in specific professional fields (such as TCM) is limited, mainly due to a lack of domain-specific expertise. Specifically, existing methods for integrating TCM knowledge into LLMs primarily rely on supervised fine-tuning (SFT), and the datasets usually contain only single-turn dialogues, lacking the ability for proactive inquiry and multi-turn dialogue understanding akin to a doctor. The paper proposes a new Chinese medical LLM model "Zhongjing," which is the first Chinese medical LLM based on the LLaMA framework, achieving a complete training process from pre-training to supervised fine-tuning to reinforcement learning from human feedback (RLHF). Additionally, the paper constructs a Chinese multi-turn medical dialogue dataset CMtMedQA, containing 70,000 real doctor-patient dialogues, significantly enhancing the model's ability to handle complex dialogues and proactive inquiries. Experimental results show that Zhongjing surpasses baseline models in multiple aspects and is comparable to ChatGPT in certain capabilities, despite having only 1% of ChatGPT's parameters. ### Main Contributions 1. **Development of a New Chinese Medical LLM**: Zhongjing is the first Chinese medical LLM to achieve a complete training process from pre-training to supervised fine-tuning to RLHF. 2. **Construction of a Multi-turn Medical Dialogue Dataset**: CMtMedQA is a multi-turn medical dialogue dataset based on 70,000 real doctor-patient dialogues, containing many instances of proactive inquiries by doctors. 3. **Improved Annotation Rules and Evaluation Standards**: The paper establishes improved annotation rules and evaluation standards for medical LLMs, customizing standard ranking annotation rules suitable for medical dialogues. 4. **Extensive Experimental Validation**: Multiple experiments were conducted on two benchmark datasets, and Zhongjing outperformed previous top Chinese medical models in all dimensions and was comparable to ChatGPT in certain areas. Through these efforts, the paper aims to enhance the performance of LLMs in the field of TCM, enabling them to better understand and respond to user needs while ensuring accuracy and safety.

Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialogue

TCMChat: A Generative Large Language Model for Traditional Chinese Medicine

MedChatZH: A tuning LLM for traditional Chinese medicine consultations

MedChatZH: a Better Medical Adviser Learns from Better Instructions

DoctorGLM: Fine-tuning your Chinese Doctor is not a Herculean Task

ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences

ChatDoctor: A Medical Chat Model Fine-tuned on LLaMA Model using Medical Domain Knowledge

Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Model

ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge

Improving Clinical Expertise in Large Language Models Using Electronic Medical Records

Enhancing Clinical Accuracy of Medical Chatbots with Large Language Models

ZhongJing: A Locally Deployed Large Language Model for Traditional Chinese Medicine and Corresponding Evaluation Methodology: A Large Language Model for data fine-tuning in the field of Traditional Chinese Medicine, and a new evaluation method called TCMEval are proposed

Lingdan: enhancing encoding of traditional Chinese medicine knowledge for clinical reasoning tasks with large language models

DoctorGPT: A Large Language Model with Chinese Medical Question-Answering Capabilities

PMC-LLaMA: Towards Building Open-source Language Models for Medicine

LLMs for Doctors: Leveraging Medical LLMs to Assist Doctors, Not Replace Them

DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultation

ZALM3: Zero-Shot Enhancement of Vision-Language Alignment via In-Context Information in Multi-Turn Multimodal Medical Dialogue

Large Language Models Leverage External Knowledge to Extend Clinical Insight Beyond Language Boundaries

AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator

Aqulia-Med LLM: Pioneering Full-Process Open-Source Medical Language Models