EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction

Siyu Yuan,Kaitao Song,Jiangjie Chen,Xu Tan,Yongliang Shen,Ren Kan,Dongsheng Li,Deqing Yang

2024-03-27

Abstract:To address intricate real-world tasks, there has been a rising interest in tool utilization in applications of large language models (LLMs). To develop LLM-based agents, it usually requires LLMs to understand many tool functions from different tool documentation. But these documentations could be diverse, redundant or incomplete, which immensely affects the capability of LLMs in using tools. To solve this, we introduce EASYTOOL, a framework transforming diverse and lengthy tool documentation into a unified and concise tool instruction for easier tool usage. EasyTool purifies essential information from extensive tool documentation of different sources, and elaborates a unified interface (i.e., tool instruction) to offer standardized tool descriptions and functionalities for LLM-based agents. Extensive experiments on multiple different tasks demonstrate that EasyTool can significantly reduce token consumption and improve the performance of tool utilization in real-world scenarios. Our code will be available at \url{

Computation and Language

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to improve the ability of agents based on large - language models (LLMs) when using external tools. Specifically, the paper points out that current tool documents have problems of inconsistency, redundancy and incompleteness, which prevent LLMs from effectively understanding and using these tools. To solve these problems, the authors propose the EASYTOOL framework, which can convert complex tool documents from different sources into unified and concise tool instructions, thus helping LLMs use tools more efficiently.

### Main contributions of the paper:
1. **Analysis and exploration**: The paper first analyzes the limitations of current LLMs - based agents in tool use and points out how the flaws in tool documents prevent LLMs from using tools.
2. **Proposing EASYTOOL**: To address the above problems, the paper proposes the EASYTOOL framework, which can extract high - quality tool instructions from tool documents to help LLMs better understand and use tools.
3. **Experimental verification**: Through experiments on multiple datasets, the paper proves that the concise tool instructions generated by EASYTOOL can significantly reduce incorrect tool use and can effectively improve the tool utilization ability of LLMs in different practical scenarios.

### Workflow of EASYTOOL:
1. **Tool description generation**: Extract the core function descriptions from the original tool documents, remove irrelevant information, and generate concise tool descriptions.
2. **Tool function guide construction**: Design a function guide for each tool, including a parameter list and usage examples, to help LLMs call tools more accurately.

### Experimental results:
- **Quality of tool descriptions**: Through the evaluation of 100 samples, it is confirmed that the generated tool descriptions have high accuracy.
- **Rationality of tool function guides**: Through the evaluation of 100 tool function guides, it is confirmed that the generated scenarios are reasonable.
- **Performance improvement**: Experiments on the ToolBench dataset show that, using the tool instructions generated by EASYTOOL, LLMs perform better in handling complex user requests, finding the correct tool solution paths, and solving complex mathematical problems.

In conclusion, this paper solves the problems of inconsistency, redundancy and incompleteness in tool documents through the EASYTOOL framework and significantly improves the tool - using ability of LLMs - based agents in practical applications.