Abstract:Large Language Models have excelled in remarkable reasoning capabilities with advanced prompting techniques, but they fall short on tasks that require exploration, strategic foresight, and sequential decision-making. Recent works propose to utilize external programs to define search logic, such that LLMs can perform passive tree search to solve more challenging reasoning tasks. Though impressive results have been achieved, there are several fundamental limitations of these approaches. First, passive tree searches are not efficient as they usually require multiple rounds of LLM API calls to solve one single problem. Moreover, passive search methods are not flexible since they need task-specific program designs. Then a natural question arises: can we maintain the tree-search capability of LLMs without the aid of external programs, and can still generate responses that clearly demonstrate the process of a tree-structure search? To this end, we propose a new concept called autonomous tree-search ability of LLM, which can automatically generate a response containing search trajectories for the correct answer. Concretely, we perform search trajectories using capable LLM API via a fixed system prompt, allowing them to perform autonomous tree-search (ATS) right out of the box. Experiments on 4 puzzle games demonstrate our method can achieve huge improvements. The ATS-BFS method outperforms the Chain of Thought approach by achieving an average accuracy improvement of 33%. Compared to Tree of Thoughts, it requires 65.6% or 47.7% less GPT-api cost to attain a comparable level of accuracy. Moreover, we have collected data using the ATS prompt method and fine-tuned LLaMA. This approach yield a greater improvement compared to the ones fine-tuned on CoT data. Specifically, it outperforms CoT-tuned LLaMAs by an average of 40.6% and 38.5% for LLaMA2-7B and LLaMA2-13B, respectively.

Implementation of Context-Sensitive L-system Based on Binary Tree

VLSI Implementation of Lempel-Ziv Binary-Tree Search Algorithm

Review on Researches of Dummy Landscape Ecology Environment Fabrication Based on L-systems

Efficient Algorithm for L-system Tree Models Based on LOD

Learning to Compose Dynamic Tree Structures for Visual Contexts

A Binary Tree Example Be Built by a Visual Method

A new method of expression and storage on decision table for the stability control of power systems

A General Method for Plants Modeling

A new BSP tree framework incorporating dynamic LoD models.

Bounded Model Checking for Branching-Time Temporal Logic

Tree-GPT: Modular Large Language Model Expert System for Forest Remote Sensing Image Understanding and Interactive Analysis

Model Checking Fuzzy Computation Tree Logic.

Simulate forest trees by integrating L-system and 3D CAD files

Tree Logic with Recursion and Model Checking Algorithm.

Visualization Of Virtual Plants Growth Based On Open L-System

Binary Tree based Chinese Word Segmentation

Implementation of Leaf Image Recognition System Based on LBP and B/S Framework

Effective Large Language Model Debugging with Best-first Tree Search

Autonomous Tree-search Ability of Large Language Models

A Tree Sampler for Bounded Context-Free Languages

Simulation for Lattice-Valued Doubly Labeled Transition Systems