Re-Search for The Truth: Multi-round Retrieval-augmented Large Language Models are Strong Fake News Detectors

Guanghua Li,Wensheng Lu,Wei Zhang,Defu Lian,Kezhong Lu,Rui Mao,Kai Shu,Hao Liao

2024-03-14

Abstract:The proliferation of fake news has had far-reaching implications on politics, the economy, and society at large. While Fake news detection methods have been employed to mitigate this issue, they primarily depend on two essential elements: the quality and relevance of the evidence, and the effectiveness of the verdict prediction mechanism. Traditional methods, which often source information from static repositories like Wikipedia, are limited by outdated or incomplete data, particularly for emerging or rare claims. Large Language Models (LLMs), known for their remarkable reasoning and generative capabilities, introduce a new frontier for fake news detection. However, like traditional methods, LLM-based solutions also grapple with the limitations of stale and long-tail knowledge. Additionally, retrieval-enhanced LLMs frequently struggle with issues such as low-quality evidence retrieval and context length constraints. To address these challenges, we introduce a novel, retrieval-augmented LLMs framework--the first of its kind to automatically and strategically extract key evidence from web sources for claim verification. Employing a multi-round retrieval strategy, our framework ensures the acquisition of sufficient, relevant evidence, thereby enhancing performance. Comprehensive experiments across three real-world datasets validate the framework's superiority over existing methods. Importantly, our model not only delivers accurate verdicts but also offers human-readable explanations to improve result interpretability.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The problem addressed in this paper is the detection of fake news. Existing methods rely on static knowledge bases or large language models, but they have issues with outdated evidence, incomplete data, and inadequate handling of emerging or rare claims. The paper proposes a novel multi-turn retrieval-enhanced large language model framework, which automatically extracts key evidence from the web to verify the authenticity of news. By performing multiple rounds of retrieval to obtain sufficient and relevant evidence, the detection performance and result interpretability are improved.