Abstract:In this paper, we introduce BiMediX, the first bilingual medical mixture of experts LLM designed for seamless interaction in both English and Arabic. Our model facilitates a wide range of medical interactions in English and Arabic, including multi-turn chats to inquire about additional details such as patient symptoms and medical history, multiple-choice question answering, and open-ended question answering. We propose a semi-automated English-to-Arabic translation pipeline with human refinement to ensure high-quality translations. We also introduce a comprehensive evaluation benchmark for Arabic medical LLMs. Furthermore, we introduce BiMed1.3M, an extensive Arabic-English bilingual instruction set covering 1.3 Million diverse medical interactions, resulting in over 632 million healthcare specialized tokens for instruction tuning. Our BiMed1.3M dataset includes 250k synthesized multi-turn doctor-patient chats and maintains a 1:2 Arabic-to-English ratio. Our model outperforms state-of-the-art Med42 and Meditron by average absolute gains of 2.5% and 4.1%, respectively, computed across multiple medical evaluation benchmarks in English, while operating at 8-times faster inference. Moreover, our BiMediX outperforms the generic Arabic-English bilingual LLM, Jais-30B, by average absolute gains of 10% on our Arabic medical benchmark and 15% on bilingual evaluations across multiple datasets. Our project page with source code and trained model is available at https://github.com/mbzuai-oryx/BiMediX .

What problem does this paper attempt to address?

The main problem this paper attempts to address is the development of a bilingual medical language model (BiMediX) that can seamlessly support both English and Arabic, thereby enhancing multilingual interaction capabilities in the medical field. Specifically, the paper addresses the following key issues: 1. **Support for Bilingual Medical Conversations**: Existing medical language models mostly support only a single language (usually English), which cannot meet the medical needs in multilingual environments. BiMediX aims to improve the quality and efficiency of cross-language medical conversations by supporting seamless switching between English and Arabic. 2. **High-Quality Translation and Dataset Construction**: To ensure the model's accuracy in a bilingual environment, the paper proposes a semi-automatic English-Arabic translation pipeline, combined with manual proofreading to generate high-quality translation data. Additionally, a bilingual dataset BiMed1.3M containing 1.3 million instructions has been constructed, covering various medical interaction scenarios such as multi-turn dialogues, multiple-choice answers, and open-ended questions. 3. **Performance Optimization and Evaluation**: The paper optimizes the Mixtral model using Parameter-Efficient Finetuning (PEFT) techniques to better adapt to medical tasks while maintaining high performance. Multiple benchmarks are introduced to evaluate BiMediX's performance in English and Arabic medical tasks, showing that BiMediX outperforms existing models on several metrics. 4. **Support for Resource-Limited Languages**: Arabic, as a resource-limited language, has relatively less application in the medical field. The paper addresses this gap by constructing a high-quality Arabic medical dataset and evaluation benchmarks, providing an important foundation for future research. In summary, by developing the BiMediX model, this paper aims to address key issues in multilingual medical conversations, enhancing the practicality and accuracy of medical language models in bilingual environments.

BiMediX: Bilingual Medical Mixture of Experts LLM

BiMediX: Bilingual Medical Mixture of Experts LLM

BiMediX2: Bio-Medical EXpert LMM for Diverse Medical Modalities

Aqulia-Med LLM: Pioneering Full-Process Open-Source Medical Language Models

MedInsight: A Multi-Source Context Augmentation Framework for Generating Patient-Centric Medical Responses using Large Language Models

Interpretable Bilingual Multimodal Large Language Model for Diverse Biomedical Tasks

Towards Evaluating and Building Versatile Large Language Models for Medicine

Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries

LLMs-in-the-loop Part-1: Expert Small AI Models for Bio-Medical Text Translation

MultiMed: Multilingual Medical Speech Recognition via Attention Encoder Decoder

Med-PMC: Medical Personalized Multi-modal Consultation with a Proactive Ask-First-Observe-Next Paradigm

CAMEL-Bench: A Comprehensive Arabic LMM Benchmark

Towards Building Multilingual Language Model for Medicine

Towards Democratizing Multilingual Large Language Models For Medicine Through A Two-Stage Instruction Fine-tuning Approach

MedExpQA: Multilingual Benchmarking of Large Language Models for Medical Question Answering

MedAide: Towards an Omni Medical Aide via Specialized LLM-based Multi-Agent Collaboration

Clinical Camel: An Open Expert-Level Medical Language Model with Dialogue-Based Knowledge Encoding

Apollo: A Lightweight Multilingual Medical LLM towards Democratizing Medical AI to 6B People

MedMobile: A mobile-sized language model with expert-level clinical capabilities

Med42-v2: A Suite of Clinical LLMs

Clinical Camel: An Open-Source Expert-Level Medical Language Model with Dialogue-Based Knowledge Encoding