Adversarial Knowledge Distillation Based Biomedical Factoid Question Answering

Jun Bai,Chuantao Yin,Jianfei Zhang,Yanmeng Wang,Yi Dong,Wenge Rong,Zhang Xiong
DOI: https://doi.org/10.1109/TCBB.2022.3161032
2023-01-01
IEEE/ACM Transactions on Computational Biology and Bioinformatics
Abstract:Biomedical factoid question answering is an essential application for biomedical information sharing. Recently, neural network based approaches have shown remarkable performance for this task. However, due to the scarcity of annotated datawhich requires intensive knowledge of expertise, training a robust model on limited-scale biomedical datasets remains a challenge. Previousworks solve this problemby introducing useful knowledge. It is found that the interaction between question and answer (QA-interaction) is also a kind of knowledge which could help extract answer accurately. This research develops a knowledge distillation framework for biomedical factoid question answering, inwhich a teacher model as the knowledge source ofQA-interaction is designed to enhance the studentmodel. In addition, to further alleviate the problemof limited-scale dataset, a novel adversarial knowledge distillation technique is proposed to robustly distill the knowledge fromteacher model to studentmodel by constructing perturbed examples as additional training data. By forcing the studentmodel to mimic the predicted distributions of teachermodel on both original examples and perturbed examples, the knowledge of QA-interaction can be learned by studentmodel. We evaluate the proposed framework on the widely used BioASQdatasets, and experimental results have shown the proposedmethod's promising potential.
What problem does this paper attempt to address?