Re-Reading Improves Reasoning in Large Language Models

Xiaohan Xu,Chongyang Tao,Tao Shen,Can Xu,Hongbo Xu,Guodong Long,Jian-guang Lou,Shuai Ma

2024-09-22

Abstract:To enhance the reasoning capabilities of off-the-shelf Large Language Models (LLMs), we introduce a simple, yet general and effective prompting method, Re2, i.e., \textbf{Re}-\textbf{Re}ading the question as input. Unlike most thought-eliciting prompting methods, such as Chain-of-Thought (CoT), which aim to elicit the reasoning process in the output, Re2 shifts the focus to the input by processing questions twice, thereby enhancing the understanding process. Consequently, Re2 demonstrates strong generality and compatibility with most thought-eliciting prompting methods, including CoT. Crucially, Re2 facilitates a "bidirectional" encoding in unidirectional decoder-only LLMs because the first pass could provide global information for the second pass. We begin with a preliminary empirical study as the foundation of Re2, illustrating its potential to enable "bidirectional" attention mechanisms. We then evaluate Re2 on extensive reasoning benchmarks across 14 datasets, spanning 112 experiments, to validate its effectiveness and generality. Our findings indicate that, with the exception of a few scenarios on vanilla ChatGPT, Re2 consistently enhances the reasoning performance of LLMs through a simple re-reading strategy. Further analyses reveal Re2's adaptability, showing how it can be effectively integrated with different LLMs, thought-eliciting prompting, and ensemble strategies. Our code is available at \url{<a class="link-external link-https" href="https://github.com/Tebmer/Rereading-LLM-Reasoning/" rel="external noopener nofollow">this https URL</a>}

Computation and Language

What problem does this paper attempt to address?

The problem this paper attempts to address is: how to enhance the reasoning capabilities of large language models (LLMs).

Specifically, existing research mainly focuses on designing various thought-provoking prompt strategies to guide LLMs in the reasoning process during the output stage, such as the "Chain-of-Thought" (CoT) method. However, these methods mostly overlook the understanding process during the input stage. The authors of the paper observe that understanding the problem is the first step in solving it, which is particularly important for complex reasoning tasks. However, most existing LLMs adopt a decoder model with a unidirectional attention mechanism (such as GPT-3 and LLaMA), which limits each token to only see the previous tokens when encoding the problem, potentially affecting the overall understanding of the problem.

To address this issue, the authors propose a simple yet effective method—RE2 (i.e., "Re-read" the problem as input). By repeating the input of the problem, RE2 enables LLMs to obtain global information when processing the problem for the second time, thereby achieving an effect similar to "bidirectional" understanding. This method not only improves the reasoning performance of LLMs but also has broad compatibility, allowing it to be combined with most thought-provoking prompt strategies (such as CoT, PAL, etc.).

The paper validates the effectiveness and generality of RE2 through experiments on multiple reasoning benchmark datasets. The results show that in most cases, RE2 can significantly enhance the reasoning performance of LLMs.