Abstract:Despite the advancements of open-source large language models (LLMs), e.g., LLaMA, they remain significantly limited in tool-use capabilities, i.e., using external tools (APIs) to fulfill human instructions. The reason is that current instruction tuning largely focuses on basic language tasks but ignores the tool-use domain. This is in contrast to the excellent tool-use capabilities of state-of-the-art (SOTA) closed-source LLMs, e.g., ChatGPT. To bridge this gap, we introduce ToolLLM, a general tool-use framework encompassing data construction, model training, and evaluation. We first present ToolBench, an instruction-tuning dataset for tool use, which is constructed automatically using ChatGPT. Specifically, the construction can be divided into three stages: (i) API collection: we collect 16,464 real-world RESTful APIs spanning 49 categories from RapidAPI Hub; (ii) instruction generation: we prompt ChatGPT to generate diverse instructions involving these APIs, covering both single-tool and multi-tool scenarios; (iii) solution path annotation: we use ChatGPT to search for a valid solution path (chain of API calls) for each instruction. To enhance the reasoning capabilities of LLMs, we develop a novel depth-first search-based decision tree algorithm. It enables LLMs to evaluate multiple reasoning traces and expand the search space. Moreover, to evaluate the tool-use capabilities of LLMs, we develop an automatic evaluator: ToolEval. Based on ToolBench, we fine-tune LLaMA to obtain an LLM ToolLLaMA, and equip it with a neural API retriever to recommend appropriate APIs for each instruction. Experiments show that ToolLLaMA demonstrates a remarkable ability to execute complex instructions and generalize to unseen APIs, and exhibits comparable performance to ChatGPT. Our ToolLLaMA also demonstrates strong zero-shot generalization ability in an out-of-distribution tool-use dataset: APIBench.

The Practice of Enhancing Learning and Scientific Innovative Abilities Using LLM-Based AI Tools

Evaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester-Long Field Study

AI for Education (AI4EDU): Advancing Personalized Education with LLM and Adaptive Learning

Large Language Models in Law: A Survey

When Young Scholars Cooperate with LLMs in Academic Tasks: The Influence of Individual Differences and Task Complexities

LLM With Tools: A Survey

Exploring the Frontiers of LLMs in Psychological Applications: A Comprehensive Review

Towards Applying Powerful Large AI Models in Classroom Teaching: Opportunities, Challenges and Prospects

LLMs for science: Usage for code generation and data analysis

LAiW: A Chinese Legal Large Language Models Benchmark

Evaluating the Impact of Advanced LLM Techniques on AI-Lecture Tutors for a Robotics Course

Creating an LLM-based AI-agent: A high-level methodology towards enhancing LLMs with APIs

Designing Child-Centric AI Learning Environments: Insights from LLM-Enhanced Creative Project-Based Learning

Navigating LLM Ethics: Advancements, Challenges, and Future Directions

Enhancing LLM-Based Feedback: Insights from Intelligent Tutoring Systems and the Learning Sciences

From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge

Large language models for automated Q&A involving legal documents: a survey on algorithms, frameworks and applications

An Empirical Study on Challenges for LLM Application Developers

ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future

TPTU: Task Planning and Tool Usage of Large Language Model-based AI Agents