Bidirectional Distillation for Multi-Guidance Medical Dialogue Generation

Qingqing Zhu,Pengfei Wu,Xiwei Wang,Dongyan Zhao,Junfei Liu
DOI: https://doi.org/10.1109/bibm52615.2021.9669534
2021-01-01
Abstract:Although researches on the dialogue system with deep learning methods have achieved good performance, medical dialogue generation confronts particular difficulties against other domains. As requiring highly accurate replies, many different types of external guidance signals are provided in previous studies to control the output and increase faithfulness. However, how these strategies compare and combine to each other is not known. In light of these challenges, we propose a multi-guidance model with bidirectional distillation for medical dialogue generation. Firstly, we fuse different guidance signals (keywords, categories and summaries) with neural sequence-to-sequence (Seq2Seq) model as teacher models. Meanwhile, we consider a simplified model without guidance as the student model. We also propose an attention mechanism to ensemble for the fusion of knowledge from multiple teachers. We further develop a bidirectional distillation module to exchange the knowledge between the teachers and a student from both sides during the training process. Through extensive experiments on medical dataset, we demonstrate the superiority of our proposed approach over state-of-the-art ones.
What problem does this paper attempt to address?