Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs

Yu Xia,Rui Wang,Xu Liu,Mingyan Li,Tong Yu,Xiang Chen,Julian McAuley,Shuai Li

2024-09-20

Abstract:Chain-of-Thought (CoT) has been a widely adopted prompting method, eliciting impressive reasoning abilities of Large Language Models (LLMs). Inspired by the sequential thought structure of CoT, a number of Chain-of-X (CoX) methods have been developed to address various challenges across diverse domains and tasks involving LLMs. In this paper, we provide a comprehensive survey of Chain-of-X methods for LLMs in different contexts. Specifically, we categorize them by taxonomies of nodes, i.e., the X in CoX, and application tasks. We also discuss the findings and implications of existing CoX methods, as well as potential future directions. Our survey aims to serve as a detailed and up-to-date resource for researchers seeking to apply the idea of CoT to broader scenarios.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that, with the wide application of the Chain - of - Thought (CoT) method in large - language models (LLMs), a series of Chain - of - X (CoX) methods have been developed to meet the challenges in different fields. However, these CoX methods have not been systematically examined or classified yet, resulting in a gap in the understanding of their potential applications. Therefore, this review article aims to provide a structured overview to capture the essence and diversity of CoX for further exploration and innovation. Specifically, the objectives of the paper include: 1. **Systematic review**: Conduct a detailed classification and discussion of the existing CoX methods, especially by organizing these methods through nodes (i.e., "X" in CoX) and application - scenario tasks. 2. **Classification method**: Classify CoX methods according to different node types (such as intermediate steps, enhancement, feedback, model, etc.), and further subdivide them according to the tasks to which they are applied. 3. **Existing findings and future directions**: Discuss the findings and impacts of the existing CoX methods and explore potential future research directions. 4. **Promote innovation**: By providing a comprehensive and up - to - date resource, help researchers apply the concept of CoT to a wider range of scenarios. In conclusion, this paper aims to fill the current gap in the understanding of CoX methods and provide a systematic framework for researchers to promote further development and innovation in this field.

Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs

ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting

Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs

Supervised Chain of Thought

Understanding Chain-of-Thought in LLMs through Information Theory

Strategic Chain-of-Thought: Guiding Accurate Reasoning in LLMs through Strategy Elicitation

How Likely Do LLMs with CoT Mimic Human Reasoning?

To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning

Self-prompted Chain-of-Thought on Large Language Models for Open-domain Multi-hop Reasoning

CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Tasks

On the Empirical Complexity of Reasoning and Planning in LLMs

Constrained Reasoning Chains for Enhancing Theory-of-Mind in Large Language Models

Chain-of-Thought Reasoning Without Prompting

Cantor: Inspiring Multimodal Chain-of-Thought of MLLM

Towards Better Chain-of-Thought Prompting Strategies: A Survey

Chain-of-Probe: Examing the Necessity and Accuracy of CoT Step-by-Step

Faithful Logical Reasoning via Symbolic Chain-of-Thought

How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning

Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters