Abstract:Tool learning enables Large Language Models (LLMs) to interact with external environments by invoking tools, serving as an effective strategy to mitigate the limitations inherent in their pre-training data. In this process, tool documentation plays a crucial role by providing usage instructions for LLMs, thereby facilitating effective tool utilization. This paper concentrates on the critical challenge of bridging the comprehension gap between LLMs and external tools due to the inadequacies and inaccuracies inherent in existing human-centric tool documentation. We propose a novel framework, DRAFT, aimed at Dynamically Refining tool documentation through the Analysis of Feedback and Trails emanating from LLMs' interactions with external tools. This methodology pivots on an innovative trial-and-error approach, consisting of three distinct learning phases: experience gathering, learning from experience, and documentation rewriting, to iteratively enhance the tool documentation. This process is further optimized by implementing a diversity-promoting exploration strategy to ensure explorative diversity and a tool-adaptive termination mechanism to prevent overfitting while enhancing efficiency. Extensive experiments on multiple datasets demonstrate that DRAFT's iterative, feedback-based refinement significantly ameliorates documentation quality, fostering a deeper comprehension and more effective utilization of tools by LLMs. Notably, our analysis reveals that the tool documentation refined via our approach demonstrates robust cross-model generalization capabilities.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the understanding gap between large - language models (LLMs) and external tools. Due to the deficiencies and inaccuracies in the existing human - centered tool documentation, LLMs encounter difficulties when using external tools. Specifically: 1. **Problems with existing tool documentation**: - **Incomplete**: Lacking key information, making it difficult for LLMs to understand the functions of tools and when to call them. - **Redundant**: Containing too much irrelevant information, increasing the token consumption in prompts and obscuring key details. - **Inaccurate**: Descriptions do not match the actual functions, which may lead to LLMs misusing tools. 2. **Limitations of manual document modification**: - **Time - consuming and labor - intensive**: Manual document modification requires a great deal of time and manpower. - **Difficult to cover all aspects**: Manual modification is difficult to comprehensively cover all usage scenarios of tools, especially edge cases. - **Difficult to update dynamically**: Tool functions are frequently updated, and it is very difficult to keep the documentation timely and accurate. To solve these problems, the paper proposes a new framework - DRAFT (Dynamically Refining tool documentation through the Analysis of Feedback and Trails), aiming to dynamically optimize tool documentation through self - driven interactions between LLMs and external tools. DRAFT achieves this goal through three main stages: 1. **Experience Gathering**: - Simulate various scenarios of tool use, generate exploration instances, and capture tool execution results. - Adopt a diversity - promoting exploration strategy to ensure the diversity and extensiveness of exploration. 2. **Learning from Experience**: - Analyze existing tool documentation, exploration instances, and tool feedback, identify problems in the documentation and make improvement suggestions. - Based on historical revision records, avoid repeating suggestions and focus on areas that still need improvement. 3. **Documentation Rewriting**: - Integrate exploration instances, tool feedback, and improvement suggestions, update tool documentation, and improve its clarity, accuracy, and usability. - Provide suggestions for the direction of the next round of exploration to form an iterative optimization process. In addition, DRAFT also introduces a tool adaptability termination mechanism, which decides when to stop modification according to the degree of documentation change, thereby improving efficiency and preventing over - fitting. Through these methods, DRAFT significantly improves the quality of tool documentation, enabling LLMs to use external tools more effectively.

From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactions

Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models

EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction

LLM With Tools: A Survey

StepTool: A Step-grained Reinforcement Learning Framework for Tool Learning in LLMs

Tool Learning with Large Language Models: A Survey

ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

Chain of Tools: Large Language Model is an Automatic Multi-tool Learner

LLMs in the Imaginarium: Tool Learning through Simulated Trial and Error

Enhancing Tool Retrieval with Iterative Feedback from Large Language Models

ToolBridge: An Open-Source Dataset to Equip LLMs with External Tool Capabilities

MetaTool: Facilitating Large Language Models to Master Tools with Meta-task Augmentation

Learning Evolving Tools for Large Language Models

Towards Completeness-Oriented Tool Retrieval for Large Language Models

Small LLMs Are Weak Tool Learners: A Multi-LLM Agent

Towards Tool Use Alignment of Large Language Models

ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios

What Affects the Stability of Tool Learning? An Empirical Study on the Robustness of Tool Learning Frameworks

Learning to Ask: When LLMs Meet Unclear Instruction