Abstract:Open-domain Question Answering models which directly leverage question-answer (QA) pairs, such as closed-book QA (CBQA) models and QA-pair retrievers, show promise in terms of speed and memory compared to conventional models which retrieve and read from text corpora. QA-pair retrievers also offer interpretable answers, a high degree of control, and are trivial to update at test time with new knowledge. However, these models lack the accuracy of retrieve-and-read systems, as substantially less knowledge is covered by the available QA-pairs relative to text corpora like Wikipedia. To facilitate improved QA-pair models, we introduce Probably Asked Questions (PAQ), a very large resource of 65M automatically-generated QA-pairs. We introduce a new QA-pair retriever, RePAQ, to complement PAQ. We find that PAQ preempts and caches test questions, enabling RePAQ to match the accuracy of recent retrieve-and-read models, whilst being significantly faster. Using PAQ, we train CBQA models which outperform comparable baselines by 5%, but trail RePAQ by over 15%, indicating the effectiveness of explicit retrieval. RePAQ can be configured for size (under 500MB) or speed (over 1K questions per second) whilst retaining high accuracy. Lastly, we demonstrate RePAQ's strength at selective QA, abstaining from answering when it is likely to be incorrect. This enables RePAQ to ``back-off" to a more expensive state-of-the-art model, leading to a combined system which is both more accurate and 2x faster than the state-of-the-art model alone.

A List Question Answering Method Based on Phrase-retrieval and Answer-ranking

Learning to Combine Answer Boundary Detection and Answer Re-ranking for Phrase-Indexed Question Answering

Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answering.

Constructing an Answer Ranking Model Using Semantic Analysis and Statistical Method for Question Answering

MrRank: Improving Question Answering Retrieval System through Multi-Result Ranking Model

Graph-Based Answer Passage Ranking for Question Answering

Term Selection and Result Reranking for Question Retrieval by Exploiting Hierarchical Classification.

Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering

Building Mutually Beneficial Relationships Between Question Retrieval And Answer Ranking To Improve Performance Of Community Question Answering

Interrogative-guided re-ranking for question-oriented software text retrieval.

Reimagining Retrieval Augmented Language Models for Answering Queries

Learning To Rank For Question-Oriented Software Text Retrieval

Algorithm for Recommending Answer Providers in Community-Based Question Answering

An Information Retrieval-Based Approach to Table-Based Question Answering.

Improving Search Relevance for Short Queries in Community Question Answering

Evaluating the Retrieval Component in LLM-Based Question Answering Systems

Optimal Answerer Ranking for New Questions in Community Question Answering

Beyond Factoid QA: Effective Methods for Non-factoid Answer Sentence Retrieval.

Capturing the Semantics of Key Phrases Using Multiple Languages for Question Retrieval

PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them

Answering Definition Question: Ranking For Top-K