Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation

Xiaoying Zhang,Baolin Peng,Ye Tian,Jingyan Zhou,Lifeng Jin,Linfeng Song,Haitao Mi,Helen Meng

2024-06-11

Abstract:Despite showing increasingly human-like abilities, large language models (LLMs) often struggle with factual inaccuracies, i.e. "hallucinations", even when they hold relevant knowledge. To address these hallucinations, current approaches typically necessitate high-quality human factuality annotations. In this work, we explore Self-Alignment for Factuality, where we leverage the self-evaluation capability of an LLM to provide training signals that steer the model towards factuality. Specifically, we incorporate Self-Eval, a self-evaluation component, to prompt an LLM to validate the factuality of its own generated responses solely based on its internal knowledge. Additionally, we design Self-Knowledge Tuning (SK-Tuning) to augment the LLM's self-evaluation ability by improving the model's confidence estimation and calibration. We then utilize these self-annotated responses to fine-tune the model via Direct Preference Optimization algorithm. We show that the proposed self-alignment approach substantially enhances factual accuracy over Llama family models across three key knowledge-intensive tasks on TruthfulQA and BioGEN.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the factual errors, namely the "hallucination" phenomenon, that occur when large - scale language models (LLMs) generate text. Although these models have demonstrated impressive capabilities, they may still generate inaccurate information even when they possess relevant knowledge. Such factual errors will undermine the credibility and practicality of LLMs in real - world applications, especially in high - risk tasks. To address this issue, the paper proposes Self - Alignment for Factuality, which uses the self - evaluation ability of LLMs to provide training signals and guide the model towards factual accuracy.

Specifically, the paper introduces two main components:
1. **SELF - EVAL**: A self - evaluation component used to prompt the LLM to verify the factuality of its generated responses, based solely on its internal knowledge.
2. **SK - TUNING**: A design for enhancing the LLM's self - evaluation ability by improving the model's confidence estimation and calibration.

Through these methods, the paper aims to reduce the factual errors of LLMs when generating text and improve their factual accuracy in key knowledge - intensive tasks such as multiple - choice question answering (MCQA), short - form open - generation, and long - form open - generation. Experimental results show that the proposed self - alignment method significantly improves the factual accuracy of the LLAMA series of models on multiple benchmark datasets.