Abstract:Objective: Recently, large language models (LLMs) have showcased remarkable capabilities in natural language understanding. While demonstrating proficiency in everyday conversations and question-answering (QA) situations, these models frequently struggle in domains that require precision, such as medical applications, due to their lack of domain-specific knowledge. In this article, we describe the procedure for building a powerful, open-source language model specifically designed for medicine applications, termed as PMC-LLaMA. Materials and methods: We adapt a general-purpose LLM toward the medical domain, involving data-centric knowledge injection through the integration of 4.8M biomedical academic papers and 30K medical textbooks, as well as comprehensive domain-specific instruction fine-tuning, encompassing medical QA, rationale for reasoning, and conversational dialogues with 202M tokens. Results: While evaluating various public medical QA benchmarks and manual rating, our lightweight PMC-LLaMA, which consists of only 13B parameters, exhibits superior performance, even surpassing ChatGPT. All models, codes, and datasets for instruction tuning will be released to the research community. Discussion: Our contributions are 3-fold: (1) we build up an open-source LLM toward the medical domain. We believe the proposed PMC-LLaMA model can promote further development of foundation models in medicine, serving as a medical trainable basic generative language backbone; (2) we conduct thorough ablation studies to demonstrate the effectiveness of each proposed component, demonstrating how different training data and model scales affect medical LLMs; (3) we contribute a large-scale, comprehensive dataset for instruction tuning. Conclusion: In this article, we systematically investigate the process of building up an open-source medical-specific LLM, PMC-LLaMA.

Model and Verification of Medical English Machine Translation Based on Optimized Generalized Likelihood Ratio Algorithm

Intelligent Recognition English Translation Model Based on Embedded Machine Learning and Improved GLR Algorithm

Uncovering Language Disparity of ChatGPT in Healthcare: Non-English Clinical Environment for Retinal Vascular Disease Classification (Preprint)

English Machine Translation Model Based on Artificial Intelligence

MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Models

LLMs-in-the-loop Part-1: Expert Small AI Models for Bio-Medical Text Translation

Efficient Fine-Tuning of Large Language Models for Automated Medical Documentation

OptimalMEE: Optimizing Large Language Models for Medical Event Extraction Through Fine-Tuning and Post-hoc Verification

Automatic Error Checking and Correction of Electronic Medical Records.

oRetrieval Augmented Generation for 10 Large Language Models and its Generalizability in Assessing Medical Fitness

ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences

Exploring Human-Like Translation Strategy with Large Language Models

Parameter-Efficient Fine-Tuning Medical Multimodal Large Language Models for Medical Visual Grounding

Development and evaluation of a large language model of ophthalmology in Chinese

PMC-LLaMA: Towards Building Open-source Language Models for Medicine

Evaluation of Language Models in the Medical Context Under Resource-Constrained Settings

PMC-LLaMA: toward building open-source language models for medicine

From Beginner to Expert: Modeling Medical Knowledge into General LLMs

Telemedicine as a special case of machine translation

Interpretable Bilingual Multimodal Large Language Model for Diverse Biomedical Tasks