Abstract:Large Language Models (LLMs) are showcasing impressive ability in handling complex reasoning tasks. In real-world situations, problems often span a spectrum of complexities. Humans inherently adjust their problem-solving approaches based on task complexity. However, most methodologies that leverage LLMs tend to adopt a uniform approach: utilizing consistent models, prompting methods, and degrees of problem decomposition, regardless of the problem complexity. Inflexibility of them can bring unnecessary computational overhead or sub-optimal performance. To address this problem, we introduce an Adaptive-Solver framework. It strategically modulates solving strategies based on the difficulties of the problems. Given an initial solution, the framework functions with two primary modules. The initial evaluation module assesses the adequacy of the current solution. If improvements are needed, the subsequent adaptation module comes into play. Within this module, three key adaptation strategies are employed: (1) Model Adaptation: Switching to a stronger LLM when a weaker variant is inadequate. (2) Prompting Method Adaptation: Alternating between different prompting techniques to suit the problem's nuances. (3) Decomposition Granularity Adaptation: Breaking down a complex problem into more fine-grained sub-questions to enhance solvability. Through such dynamic adaptations, our framework not only enhances computational efficiency but also elevates the overall performance. This dual-benefit ensures both the efficiency of the system for simpler tasks and the precision required for more complex questions. Experimental results from complex reasoning tasks reveal that the prompting method adaptation and decomposition granularity adaptation enhance performance across all tasks. Furthermore, the model adaptation approach significantly reduces API costs (up to 50%) while maintaining superior performance.

A Model and Heuristic For Solving Very Large Item Selection Problems

Novel item selection strategies for cognitive diagnostic computerized adaptive testing: A heuristic search framework

Modeling Item-Level Heterogeneous Treatment Effects With the Explanatory Item Response Model: Leveraging Large-Scale Online Assessments to Pinpoint the Impact of Educational Interventions

A Heuristic Lot-Sizing Algorithm for Large Scale Lot-Sizing Problem

Data-driven Branching and Selection for Lot-Sizing and Scheduling Problems with Sequence-Dependent Setups and Setup Carryover

Efficient dynamic model based testing using greedy test case selection

Large-P Variable Selection in Two-Stage Models

Selection hyper-heuristics and job shop scheduling problems: How does instance size influence performance?

Optimal Design for Item Calibration in Computerized Adaptive Testing: the 2PL Case

Unit Selection: Case Study and Comparison with A/B Test Heuristic

Analytics Branching and Selection for the Capacitated Multi-Item Lot Sizing Problem with Nonidentical Machines

Nonlinear Sequential Designs for Logistic Item Response Theory Models with Applications to Computerized Adaptive Tests

Testing the Robustness of Linear Programming* Using a Diet Problem on a Multi-Shop System

A Comparison of Metaheuristic Optimization Algorithms for Scale Short-Form Development

Will the Real Linda Please Stand up...to Large Language Models? Examining the Representativeness Heuristic in LLMs

Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model

Distance and Matching-Induced Search Algorithm for the Multi-Level Lot-Sizing Problem with Substitutable Bill of Materials

A Hidden Markov Model Approach to the Problem of Heuristic Selection in Hyper-Heuristics with a Case Study in High School Timetabling Problems

Scalable Learning of Item Response Theory Models

A Problem‐Solving Tool for Single‐Case Design Selection in Practice

Adaptive-Solver Framework for Dynamic Strategy Selection in Large Language Model Reasoning