Abstract:Large Language Models (LLMs) are showcasing impressive ability in handling complex reasoning tasks. In real-world situations, problems often span a spectrum of complexities. Humans inherently adjust their problem-solving approaches based on task complexity. However, most methodologies that leverage LLMs tend to adopt a uniform approach: utilizing consistent models, prompting methods, and degrees of problem decomposition, regardless of the problem complexity. Inflexibility of them can bring unnecessary computational overhead or sub-optimal performance. To address this problem, we introduce an Adaptive-Solver framework. It strategically modulates solving strategies based on the difficulties of the problems. Given an initial solution, the framework functions with two primary modules. The initial evaluation module assesses the adequacy of the current solution. If improvements are needed, the subsequent adaptation module comes into play. Within this module, three key adaptation strategies are employed: (1) Model Adaptation: Switching to a stronger LLM when a weaker variant is inadequate. (2) Prompting Method Adaptation: Alternating between different prompting techniques to suit the problem's nuances. (3) Decomposition Granularity Adaptation: Breaking down a complex problem into more fine-grained sub-questions to enhance solvability. Through such dynamic adaptations, our framework not only enhances computational efficiency but also elevates the overall performance. This dual-benefit ensures both the efficiency of the system for simpler tasks and the precision required for more complex questions. Experimental results from complex reasoning tasks reveal that the prompting method adaptation and decomposition granularity adaptation enhance performance across all tasks. Furthermore, the model adaptation approach significantly reduces API costs (up to 50%) while maintaining superior performance.

ANSWERED: Adaptive Tool-Augmented LLMs with Strategic Error Feedback for Compositional Reasoning

Concise and Organized Perception Facilitates Large Language Models for Deductive Reasoning.

Understanding and Patching Compositional Reasoning in LLMs

Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models

Adaptive-Solver Framework for Dynamic Strategy Selection in Large Language Model Reasoning

MAF: Multi-Aspect Feedback for Improving Reasoning in Large Language Models

Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalability

Exploring the Compositional Deficiency of Large Language Models in Mathematical Reasoning

Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering

Look Before You Decide: Prompting Active Deduction of MLLMs for Assumptive Reasoning

Can We Further Elicit Reasoning in LLMs? Critic-Guided Planning with Retrieval-Augmentation for Solving Challenging Tasks

Retrieval Meets Reasoning: Dynamic In-Context Editing for Long-Text Understanding

Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning

An Enhanced Prompt-Based LLM Reasoning Scheme via Knowledge Graph-Integrated Collaboration

Federated Prompting and Chain-of-Thought Reasoning for Improving LLMs Answering

Concise and Organized Perception Facilitates Reasoning in Large Language Models

Enhancing Language Model Reasoning via Weighted Reasoning in Self-Consistency

ALR$^2$: A Retrieve-then-Reason Framework for Long-context Question Answering

Make LLMs better zero-shot reasoners: Structure-orientated autonomous reasoning

CLR-Fact: Evaluating the Complex Logical Reasoning Capability of Large Language Models over Factual Knowledge