Abstract:Large Language Models (LLMs) have recently made significant strides in complex reasoning tasks through the Chain-of-Thought technique. Despite this progress, their reasoning is often constrained by their intrinsic understanding, lacking external insights. To address this, we propose Exchange-of-Thought (EoT), a novel framework that enables cross-model communication during problem-solving. Drawing inspiration from network topology, EoT integrates four unique communication paradigms: Memory, Report, Relay, and Debate. This paper delves into the communication dynamics and volume associated with each paradigm. To counterbalance the risks of incorrect reasoning chains, we implement a robust confidence evaluation mechanism within these communications. Our experiments across diverse complex reasoning tasks demonstrate that EoT significantly surpasses established baselines, underscoring the value of external insights in enhancing LLM performance. Furthermore, we show that EoT achieves these superior results in a cost-effective manner, marking a promising advancement for efficient and collaborative AI problem-solving.

What problem does this paper attempt to address?

This paper attempts to address the issue of large language models (LLMs) performing poorly on complex reasoning tasks. Despite significant progress made by LLMs in complex reasoning tasks through the Chain-of-Thought (CoT) technique, their reasoning abilities are still limited by intrinsic understanding and lack external insights. To tackle this problem, the paper proposes the Exchange-of-Thought (EoT) framework, which introduces external insights into the problem-solving process through cross-model communication. Specifically, the EoT framework includes the following points: 1. **Cross-Model Communication**: EoT facilitates information exchange between models through four unique communication paradigms (Memory, Report, Relay, and Debate), drawing principles from network topology and intelligent agent communication. 2. **Communication Dynamics and Volume**: The paper provides a detailed analysis of the communication dynamics and volume for each communication paradigm to optimize information flow and processing efficiency. 3. **Confidence Assessment Mechanism**: To balance the risk of erroneous reasoning chains, EoT introduces a confidence assessment mechanism that evaluates the model's confidence level by analyzing changes in answers, thereby reducing the impact of incorrect reasoning. 4. **Experimental Validation**: The paper validates the effectiveness and cost-efficiency of the EoT framework through experiments on various complex reasoning tasks, demonstrating the importance of external insights in enhancing LLM performance. In summary, this paper aims to enhance the performance of LLMs in complex reasoning tasks by introducing cross-model communication and external insights.

Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication

Concise and Organized Perception Facilitates Large Language Models for Deductive Reasoning.

Synergy-of-Thoughts: Eliciting Efficient Reasoning in Hybrid Language Models

Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaboration

Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning

Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models

Thought Space Explorer: Navigating and Expanding Thought Space for Large Language Model Reasoning

Towards Faithful Chain-of-Thought: Large Language Models are Bridging Reasoners

Logic-of-Thought: Injecting Logic into Contexts for Full Reasoning in Large Language Models

CLR-Fact: Evaluating the Complex Logical Reasoning Capability of Large Language Models over Factual Knowledge

Chain-of-Experts: when LLMs Meet Complex Operations Research Problems

Large Language Models Need Consultants for Reasoning: Becoming an Expert in a Complex Human System Through Behavior Simulation

Enhancing Zero-Shot Chain-of-Thought Reasoning in Large Language Models through Logic

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model with Knowledge Graph

Evolution of Thought: Diverse and High-Quality Reasoning via Multi-Objective Optimization

Empowering Multi-step Reasoning across Languages via Tree-of-Thoughts

Towards Reasoning in Large Language Models via Multi-Agent Peer Review Collaboration

Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation

Mind's Mirror: Distilling Self-Evaluation Capability and Comprehensive Thinking from Large Language Models

ToM-LM: Delegating Theory of Mind Reasoning to External Symbolic Executors in Large Language Models