Abstract:Recent state-of-the-art open-domain QA models are typically based on a two stage retriever-reader approach in which the retriever first finds the relevant knowledge/passages and the reader then leverages that to predict the answer. Prior work has shown that the performance of the reader usually tends to improve with the increase in the number of these passages. Thus, state-of-the-art models use a large number of passages (e.g. 100) for inference. While the reader in this approach achieves high prediction performance, its inference is computationally very expensive. We humans, on the other hand, use a more efficient strategy while answering: firstly, if we can confidently answer the question using our already acquired knowledge then we do not even use the external knowledge, and in the case when we do require external knowledge, we don't read the entire knowledge at once, instead, we only read that much knowledge that is sufficient to find the answer. Motivated by this procedure, we ask a research question "Can the open-domain QA reader utilize external knowledge efficiently like humans without sacrificing the prediction performance?" Driven by this question, we explore an approach that utilizes both 'closed-book' (leveraging knowledge already present in the model parameters) and 'open-book' inference (leveraging external knowledge). Furthermore, instead of using a large fixed number of passages for open-book inference, we dynamically read the external knowledge in multiple 'knowledge iterations'. Through comprehensive experiments on NQ and TriviaQA datasets, we demonstrate that this dynamic reading approach improves both the 'inference efficiency' and the 'prediction accuracy' of the reader. Comparing with the FiD reader, this approach matches its accuracy by utilizing just 18.32% of its reader inference cost and also outperforms it by achieving up to 55.10% accuracy on NQ Open.

Careful Selection of Knowledge to solve Open Book Question Answering

Can Open-Domain QA Reader Utilize External Knowledge Efficiently like Humans?

Knowledge-Aided Open-Domain Question Answering

Coarse-to-Careful: Seeking Semantic-related Knowledge for Open-domain Commonsense Question Answering

Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Study

Can Generative Pre-trained Language Models Serve As Knowledge Bases for Closed-book QA?

SKR-QA: Semantic ranking and knowledge revise for multi-choice question answering

KBQA: Learning Question Answering over QA Corpora and Knowledge Bases

Open Domain Question Answering System Based on Knowledge Base.

Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering

Answering Science Exam Questions Using Query Rewriting with Background Knowledge

Leveraging Knowledge Graph for Open-Domain Question Answering

CBR-iKB: A Case-Based Reasoning Approach for Question Answering over Incomplete Knowledge Bases

Do I have the Knowledge to Answer? Investigating Answerability of Knowledge Base Questions

BB-KBQA: BERT-Based Knowledge Base Question Answering

Benchmarking Knowledge-Enhanced Commonsense Question Answering via Knowledge-to-Text Transformation

Knowledge Fusion and Semantic Knowledge Ranking for Open Domain Question Answering

Open Domain Question Answering with A Unified Knowledge Interface

Knowledge-Enhanced Retrieval: A Scheme for Question Answering

Gs-Cbr-Kbqa: Graph-Structured Case-Based Reasoning for Knowledge Base Question Answering

Querying Knowledge via Multi-Hop English Questions