Abstract:Current large language models (LLMs) can exhibit near-human levels of performance on many natural language-based tasks, including open-domain question answering. Unfortunately, at this time, they also convincingly hallucinate incorrect answers, so that responses to questions must be verified against external sources before they can be accepted at face value. In this paper, we report two simple experiments to automatically validate generated answers against a corpus. We base our experiments on questions and passages from the MS MARCO (V1) test collection, and a retrieval pipeline consisting of sparse retrieval, dense retrieval and neural rerankers. In the first experiment, we validate the generated answer in its entirety. After presenting a question to an LLM and receiving a generated answer, we query the corpus with the combination of the question + generated answer. We then present the LLM with the combination of the question + generated answer + retrieved answer, prompting it to indicate if the generated answer can be supported by the retrieved answer. In the second experiment, we consider the generated answer at a more granular level, prompting the LLM to extract a list of factual statements from the answer and verifying each statement separately. We query the corpus with each factual statement and then present the LLM with the statement and the corresponding retrieved evidence. The LLM is prompted to indicate if the statement can be supported and make necessary edits using the retrieved material. With an accuracy of over 80%, we find that an LLM is capable of verifying its generated answer when a corpus of supporting material is provided. However, manual assessment of a random sample of questions reveals that incorrect generated answers are missed by this verification process. While this verification process can reduce hallucinations, it can not entirely eliminate them.

Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering

Generator-Retriever-Generator Approach for Open-Domain Question Answering

A Hybrid Text Generation-Based Query Expansion Method for Open-Domain Question Answering

A Copy-Augmented Generative Model for Open-Domain Question Answering

Modeling Uncertainty and Using Post-fusion as Fallback Improves Retrieval Augmented Generation with LLMs

Adapting Pre-trained Generative Models for Extractive Question Answering

Generation-Augmented Retrieval for Open-domain Question Answering

Retrieving Supporting Evidence for Generative Question Answering

To Generate or to Retrieve? On the Effectiveness of Artificial Contexts for Medical Open-Domain Question Answering

GenSco: Can Question Decomposition based Passage Alignment improve Question Answering?

How Does Generative Retrieval Scale to Millions of Passages?

Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering

You Only Need One Model for Open-domain Question Answering

UnitedQA: A Hybrid Approach for Open Domain Question Answering

QPaug: Question and Passage Augmentation for Open-Domain Question Answering of LLMs

FiE: Building a Global Probability Space by Leveraging Early Fusion in Encoder for Open-Domain Question Answering

Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering

GripRank: Bridging the Gap between Retrieval and Generation via the Generative Knowledge Improved Passage Ranking

UniGen: A Unified Generative Framework for Retrieval and Question Answering with Large Language Models

Understanding and Improving Zero-shot Multi-hop Reasoning in Generative Question Answering