ToolBridge: An Open-Source Dataset to Equip LLMs with External Tool Capabilities

Zhenchao Jin,Mengchen Liu,Dongdong Chen,Lingting Zhu,Yunsheng Li,Lequan Yu

2024-10-09

Abstract:Through the integration of external tools, large language models (LLMs) such as GPT-4o and Llama 3.1 significantly expand their functional capabilities, evolving from elementary conversational agents to general-purpose assistants. We argue that the primary drivers of these advancements are the quality and diversity of the training data. However, the existing LLMs with external tool integration provide only limited transparency regarding their datasets and data collection methods, which has led to the initiation of this research. Specifically, in this paper, our objective is to elucidate the detailed process involved in constructing datasets that empower LLMs to effectively learn how to utilize external tools and make this information available to the public through the introduction of ToolBridge. ToolBridge proposes to employ a collection of general open-access datasets as its raw dataset pool and applies a series of strategies to identify appropriate data entries from the pool for external tool API insertions. By supervised fine-tuning on these curated data entries, LLMs can invoke external tools in appropriate contexts to boost their predictive accuracy, particularly for basic functions including data processing, numerical computation, and factual retrieval. Our experiments rigorously isolates model architectures and training configurations, focusing exclusively on the role of data. The experimental results indicate that LLMs trained on ToolBridge demonstrate consistent performance improvements on both standard benchmarks and custom evaluation datasets. All the associated code and data will be open-source at <a class="link-external link-https" href="https://github.com/CharlesPikachu/ToolBridge" rel="external noopener nofollow">this https URL</a>, promoting transparency and facilitating the broader community to explore approaches for equipping LLMs with external tools capabilities.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the insufficient ability of large - language models (LLMs) in using external tools, especially their poor performance in basic functions such as arithmetic operations and fact - finding. Although existing research has attempted to enhance the functions of LLMs by introducing external tools, these studies often lack transparency, especially with regard to the sources of training data and the construction methods not being made public. This restricts the community's further exploration and development in this area.

For this reason, this paper proposes a dataset construction pipeline named ToolBridge, aiming to create high - quality, publicly available benchmark datasets for training LLMs on how to effectively use external tools. Specifically, ToolBridge screens, transforms, and filters valuable entries from open - source datasets and finally generates a dataset with more than 178,000 data entries. These data entries are designed to guide LLMs to call external tools in appropriate situations, thereby improving their prediction accuracy, especially in basic functions such as data processing, numerical calculation, and fact retrieval.

Through this method, ToolBridge not only improves the performance of LLMs but also shows significant improvements on standard benchmark tests and custom - evaluation datasets. In addition, the author has also open - sourced all relevant code and data, promoting transparency in this field and providing the broader community with the opportunity to explore how to equip LLMs with the ability to use external tools.