Abstract:Chain-of-thought (CoT) advances the reasoning abilities of large language models (LLMs) and achieves superior performance in complex reasoning tasks. However, most CoT studies rely on carefully designed human-annotated rational chains to prompt LLMs, posing challenges for real-world applications where labeled data is available without rational chains. This paper proposes a new strategy, Automate-CoT (Automatic Prompt Augmentation and Selection with Chain-of-Thought), that can bypass human engineering of CoT by automatically augmenting rational chains from a small labeled dataset, and then pruning low-quality chains to construct a candidate pool of machine-generated rationale chains based on the labels. Finally, it selects the optimal combination of several rationale chains from the pool for CoT prompting by employing a variance-reduced policy gradient strategy to estimate the significance of each example. Automate-CoT enables a quick adaptation of the CoT technique to different tasks. Experimental results demonstrate the effectiveness of our method, where competitive results are achieved on arithmetic reasoning (+2.7%), commonsense reasoning (+3.4%), symbolic reasoning (+3.2%), and non-reasoning tasks (+2.5%). The code is available at <a class="link-external link-https" href="https://github.com/SHUMKASHUN/Automate-CoT" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The problem this paper attempts to address is: how to automatically enhance and select appropriate Chain-of-Thought (CoT) examples without the need for manual design, in order to improve the performance of large language models (LLMs) in complex reasoning tasks. Specifically, the paper proposes a new strategy—Automate-CoT (Automated Chain-of-Thought Augmentation and Selection), which aims to solve the manual dependency issue in existing methods through the following steps: 1. **Augment**: Automatically generate multiple pseudo Chain-of-Thought examples using a language model, based on a small annotated dataset. 2. **Prune**: Eliminate low-quality Chain-of-Thought examples based on the consistency between the generated answers and the true answers, thereby constructing a high-quality machine-generated Chain-of-Thought pool. 3. **Select**: Use a variance-reduced policy gradient strategy to select the optimal combination of Chain-of-Thought examples from the pool to optimize the performance for each task. The paper points out that existing Chain-of-Thought methods often rely on carefully designed manually annotated reasoning chains, which pose challenges in practical applications, especially when the annotated data does not include reasoning chains. Automate-CoT overcomes these issues by automatically generating and selecting Chain-of-Thought examples, improving performance in arithmetic reasoning, commonsense reasoning, symbolic reasoning, and non-reasoning tasks. Experimental results show that Automate-CoT achieves significant performance improvements across multiple tasks, such as a 2.7% improvement in arithmetic reasoning tasks, a 3.4% improvement in commonsense reasoning tasks, a 3.2% improvement in symbolic reasoning tasks, and a 2.5% improvement in non-reasoning tasks.

Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data

Automatic prompt augmentation and selection with chain-of-thought from labeled data

ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting

Automatic Chain of Thought Prompting in Large Language Models

Active Prompting with Chain-of-Thought for Large Language Models

Boosting Language Models Reasoning with Chain-of-Knowledge Prompting

Chain-of-Thought Reasoning Without Prompting

Towards understanding chain-of-thought prompting: An empirical study of what matters

Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs

Supervised Chain of Thought

ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models

Pattern-Aware Chain-of-Thought Prompting in Large Language Models

A comparison of chain-of-thought reasoning strategies across datasets and models

Constrained Reasoning Chains for Enhancing Theory-of-Mind in Large Language Models

Strategic Chain-of-Thought: Guiding Accurate Reasoning in LLMs through Strategy Elicitation

An automatically discovered chain-of-thought prompt generalizes to novel models and datasets

To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning