Abstract: Biomedical Question Answering (BQA) has attracted increasing attention in recent years due to its promising application prospect. It is a challenging task because the biomedical questions are professional and usually vary widely. Existing question answering methods answer all questions with a homogeneous model, leading to various types of questions competing for the shared parameters, which will confuse the model decision for each single type of questions. In this paper, in order to alleviate the parameter competition problem, we propose a Mixture-of-Expert (MoE) based question answering method called MoEBQA that decouples the computation for different types of questions by sparse routing. To be specific, we split a pretrained Transformer model into bottom and top blocks. The bottom blocks are shared by all the examples, aiming to capture the general features. The top blocks are extended to an MoE version that consists of a series of independent experts, where each example is assigned to a few experts according to its underlying question type. MoEBQA automatically learns the routing strategy in an end-to-end manner so that each expert tends to deal with the question types it is expert in. We evaluate MoEBQA on three BQA datasets constructed based on real examinations. The results show that our MoE extension significantly boosts the performance of question answering models and achieves new state-of-the-art performance. In addition, we elaborately analyze our MoE modules to reveal how MoEBQA works and find that it can automatically group the questions into human-readable clusters.

Improving Biomedical Question Answering by Data Augmentation and Model Weighting

Contextual embedding and model weighting by fusing domain knowledge on Biomedical Question Answering

Optimized Biomedical Question-Answering Services with LLM and Multi-BERT Integration

External features enriched model for biomedical question answering

PubMedQA: A Dataset for Biomedical Research Question Answering

Biomedical Question Answering: A Survey of Approaches and Challenges

DKE-Research at SemEval-2024 Task 2: Incorporating Data Augmentation with Generative Models and Biomedical Knowledge to Enhance Inference Robustness

XAIQA: Explainer-Based Data Augmentation for Extractive Question Answering

Unsupervised Pre-training for Biomedical Question Answering

Keyword-based Data Augmentation Guided Chinese Medical Questions Classification

Pre-trained models, data augmentation, and ensemble learning for biomedical information extraction and document classification

Mixture of Experts for Biomedical Question Answering

Medical Data Inquiry Using a Question Answering Model.

Efficient Medical Question Answering with Knowledge-Augmented Question Generation

Neural Domain Adaptation for Biomedical Question Answering

How to Pre-Train Your Model? Comparison of Different Pre-Training Models for Biomedical Question Answering

Question Answering With Character-Level Lstm Encoders And Model-Based Data Augmentation

Asymmetric cross-modal attention network with multimodal augmented mixup for medical visual question answering

Using Pretrained Large Language Model with Prompt Engineering to Answer Biomedical Questions