Abstract:Recent state-of-the-art open-domain QA models are typically based on a two stage retriever-reader approach in which the retriever first finds the relevant knowledge/passages and the reader then leverages that to predict the answer. Prior work has shown that the performance of the reader usually tends to improve with the increase in the number of these passages. Thus, state-of-the-art models use a large number of passages (e.g. 100) for inference. While the reader in this approach achieves high prediction performance, its inference is computationally very expensive. We humans, on the other hand, use a more efficient strategy while answering: firstly, if we can confidently answer the question using our already acquired knowledge then we do not even use the external knowledge, and in the case when we do require external knowledge, we don't read the entire knowledge at once, instead, we only read that much knowledge that is sufficient to find the answer. Motivated by this procedure, we ask a research question "Can the open-domain QA reader utilize external knowledge efficiently like humans without sacrificing the prediction performance?" Driven by this question, we explore an approach that utilizes both 'closed-book' (leveraging knowledge already present in the model parameters) and 'open-book' inference (leveraging external knowledge). Furthermore, instead of using a large fixed number of passages for open-book inference, we dynamically read the external knowledge in multiple 'knowledge iterations'. Through comprehensive experiments on NQ and TriviaQA datasets, we demonstrate that this dynamic reading approach improves both the 'inference efficiency' and the 'prediction accuracy' of the reader. Comparing with the FiD reader, this approach matches its accuracy by utilizing just 18.32% of its reader inference cost and also outperforms it by achieving up to 55.10% accuracy on NQ Open.

Open Domain Question Answering Based on Retriever-Reader Architecture

Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering

End-to-End Training of Neural Retrievers for Open-Domain Question Answering

Domain-specific Cross-Language Relevant Question Retrieval.

L2R-QA: An Open-Domain Question Answering Framework

Mutually improved dense retriever and GNN-based reader for arbitrary-hop open-domain question answering

R-3: Reinforced Ranker-Reader for Open-Domain Question Answering

A Survey for Efficient Open Domain Question Answering

R$^3$: Reinforced Reader-Ranker for Open-Domain Question Answering

Unsupervised Open-Domain Question Answering

Document Gated Reader for Open-Domain Question Answering

SS-BERT: A Semantic Information Selecting Approach for Open-Domain Question Answering

Low-Resource Dense Retrieval for Open-Domain Question Answering: A Comprehensive Survey

OpenQA: Hybrid QA System Relying on Structured Knowledge Base As Well As Non-structured Data

A Chinese Intelligent Question Answering System Based on Domain Ontology and Sentence Templates

Advances in Open-Domain Question Answering

Can Open-Domain QA Reader Utilize External Knowledge Efficiently like Humans?

Dense Hierarchical Retrieval for Open-Domain Question Answering

Open Domain Question Answering with Character-Level Deep Learning Models.

A Multimodal Dense Retrieval Approach for Speech-Based Open-Domain Question Answering

An Open Domain Question Answering System Based on Improved System Similarity Model