Abstract:Question Answering (QA) systems have been extensively studied in both academia and the research community due to their wide real-world applications. When building such industrial-scale QA applications, we are facing two prominent challenges, i.e., i) lacking a sufficient amount of training data to learn an accurate model and ii) requiring high inference speed for online model serving. There are generally two ways to mitigate the above-mentioned problems. One is to adopt transfer learning to leverage information from other domains; the other is to distill the “dark knowledge” from a large teacher model to small student models. The former usually employs parameter sharing mechanisms for knowledge transfer, but does not utilize the “dark knowledge” of pre-trained large models. The latter usually does not consider the cross-domain information from other domains. We argue that these two types of methods can be complementary to each other. Hence in this work, we provide a new perspective on the potential of the teacher-student paradigm facilitating cross-domain transfer learning, where the teacher and student tasks belong to heterogeneous domains, with the goal to improve the student model’s performance in the target domain. Our framework considers the “dark knowledge” learned from large teacher models and also leverages the adaptive hints to alleviate the domain differences between teacher and student models. Extensive experiments have been conducted on two text matching tasks for retrieval-based QA systems. Results show the proposed method has better performance than the competing methods including the existing state-of-the-art transfer learning methods. We have also deployed our method in an online production system and observed significant improvements compared to the existing approaches in terms of both accuracy and cross-domain robustness.

Interactive Model and Application of Joint Knowledge Base Question Answering and Semantic Matching.

An Intelligent Question Answering System for University Courses Based on BiLSTM and Keywords Similarity.

A Chinese Intelligent Question Answering System Based on Domain Ontology and Sentence Templates

Dynamic Multi Teacher Knowledge Distillation for Semantic Parsing in Kbqa

ComQA: Question Answering over Knowledge Base Via Semantic Matching.

Topic enhanced deep structured semantic models for knowledge base question answering

BB-KBQA: BERT-Based Knowledge Base Question Answering

An Intelligent Question-Answering Model over Educational Knowledge Graph for Sustainable Urban Living

Constructing a Hybrid Automatic Q&A System Integrating Knowledge Graph and Information Retrieval Technologies

Complex Knowledge Base Question Answering for Intelligent Bridge Management Based on Multi-Task Learning and Cross-Task Constraints

Question-aware Graph Convolutional Network for Educational Knowledge Base Question Answering

Knowledge-Enhanced Retrieval: A Scheme for Question Answering

A Survey on Complex Knowledge Base Question Answering: Methods, Challenges and Solutions

Gs-Cbr-Kbqa: Graph-Structured Case-Based Reasoning for Knowledge Base Question Answering

Cross-domain Knowledge Distillation for Retrieval-based Question Answering Systems.

Interactive-KBQA: Multi-Turn Interactions for Knowledge Base Question Answering with Large Language Models

Knowledge Base Question Answering with a Matching-Aggregation Model and Question-Specific Contextual Relations

Knowledge Base Question Answering with Topic Units

SKR-QA: Semantic ranking and knowledge revise for multi-choice question answering

ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models

KBQA: Learning Question Answering over QA Corpora and Knowledge Bases