Abstract:There emerges a promising trend of using large language models (LLMs) to generate code-like plans for complex inference tasks such as visual reasoning. This paradigm, known as LLM-based planning, provides flexibility in problem solving and endows better interpretability. However, current research is mostly limited to basic scenarios of simple questions that can be straightforward answered in a few inference steps. Planning for the more challenging multi-hop visual reasoning tasks remains under-explored. Specifically, under multi-hop reasoning situations, the trade-off between accuracy and the complexity of plan-searching becomes prominent. The prevailing algorithms either address the efficiency issue by employing the fast one-stop generation or adopt a complex iterative generation method to improve accuracy. Both fail to balance the need for efficiency and performance. Drawing inspiration from the dual system of cognition in the human brain, the fast and the slow think processes, we propose a hierarchical plan-searching algorithm that integrates the one-stop reasoning (fast) and the Tree-of-thought (slow). Our approach succeeds in performance while significantly saving inference steps. Moreover, we repurpose the PTR and the CLEVER datasets, developing a systematic framework for evaluating the performance and efficiency of LLMs-based plan-search algorithms under reasoning tasks at different levels of difficulty. Extensive experiments demonstrate the superiority of our proposed algorithm in terms of performance and efficiency. The dataset and code will be release soon.

SEED: Accelerating Reasoning Tree Construction via Scheduled Speculative Decoding

Concise and Organized Perception Facilitates Large Language Models for Deductive Reasoning.

Graph-Structured Speculative Decoding

Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding

Cascade Speculative Drafting for Even Faster LLM Inference

SSSD: Simply-Scalable Speculative Decoding

Tree-of-Mixed-Thought: Combining Fast and Slow Thinking for Multi-hop Visual Reasoning

Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding

DySpec: Faster Speculative Decoding with Dynamic Token Tree Structure

Dynamic Depth Decoding: Faster Speculative Decoding for LLMs

Decoding Speculative Decoding

Recursive Speculative Decoding: Accelerating LLM Inference via Sampling Without Replacement

ParallelSpec: Parallel Drafter for Efficient Speculative Decoding

SuffixDecoding: A Model-Free Approach to Speeding Up Large Language Model Inference

SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices

Online Speculative Decoding

Accelerating LLM Inference with Staged Speculative Decoding

Faster Cascades via Speculative Decoding

Deductive Beam Search: Decoding Deducible Rationale for Chain-of-Thought Reasoning

DeFT: Flash Tree-attention with IO-Awareness for Efficient Tree-search-based LLM Inference.