Abstract:Large Language Models (LLMs) exhibit remarkable proficiency in addressing a diverse array of tasks within the Natural Language Processing (NLP) domain, with various prompt design strategies significantly augmenting their capabilities. However, these prompts, while beneficial, each possess inherent limitations. The primary prompt design methodologies are twofold: The first, exemplified by the Chain of Thought (CoT), involves manually crafting prompts specific to individual datasets, hence termed Expert-Designed Prompts (EDPs). Once these prompts are established, they are unalterable, and their effectiveness is capped by the expertise of the human designers. When applied to LLMs, the static nature of EDPs results in a uniform approach to both simple and complex problems within the same dataset, leading to the inefficient use of tokens for straightforward issues. The second method involves prompts autonomously generated by the LLM, known as LLM-Derived Prompts (LDPs), which provide tailored solutions to specific problems, mitigating the limitations of EDPs. However, LDPs may encounter a decline in performance when tackling complex problems due to the potential for error accumulation during the solution planning process. To address these challenges, we have conceived a novel Prompt Recursive Search (PRS) framework that leverages the LLM to generate solutions specific to the problem, thereby conserving tokens. The framework incorporates an assessment of problem complexity and an adjustable structure, ensuring a reduction in the likelihood of errors. We have substantiated the efficacy of PRS framework through extensive experiments using LLMs with different numbers of parameters across a spectrum of datasets in various domains. Compared to the CoT method, the PRS method has increased the accuracy on the BBH dataset by 8% using Llama3-7B model, achieving a 22% improvement.

Local Search-based Approach for Cost-effective Job Assignment on Large Language Models.

OptLLM: Optimal Assignment of Queries to Large Language Models

Studying and Benchmarking Large Language Models For Log Level Suggestion

Cost-Effective Online Multi-LLM Selection with Versatile Reward Models

On Speeding Up Language Model Evaluation

Automatic Prompt Selection for Large Language Models

Search for Efficient Large Language Models

Prompt Recursive Search: A Living Framework with Adaptive Growth in LLM Auto-Prompting

The economic trade-offs of large language models: A case study

SelectLLM: Query-Aware Efficient Selection Algorithm for Large Language Models

LogEval: A Comprehensive Benchmark Suite for Large Language Models In Log Analysis

SMART: Automatically Scaling Down Language Models with Accuracy Guarantees for Reduced Processing Fees

Optimizing Search-Based Unit Test Generation with Large Language Models: an Empirical Study

Towards Optimizing with Large Language Models

Strategic Optimization and Challenges of Large Language Models in Object-Oriented Programming

MetaLLM: A High-performant and Cost-efficient Dynamic Framework for Wrapping LLMs

Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templates

Efficient LLM Scheduling by Learning to Rank

Active Testing of Large Language Model via Multi-Stage Sampling

Optimizing Numerical Estimation and Operational Efficiency in the Legal Domain through Large Language Models

Facilitating Multi-Role and Multi-Behavior Collaboration of Large Language Models for Online Job Seeking and Recruiting