Meta-Reasoning Improves Tool Use in Large Language Models

Lisa Alazraki,Marek Rei

2024-11-07

Abstract:External tools help large language models (LLMs) succeed at tasks where they would otherwise typically fail. In existing frameworks, LLMs learn tool use either by in-context demonstrations or via full model fine-tuning on annotated data. As these approaches do not easily scale, a recent trend is to abandon them in favor of lightweight, parameter-efficient tuning paradigms. These methods allow quickly alternating between the frozen LLM and its specialised fine-tuned version, by switching on or off a handful of additional custom parameters. Hence, we postulate that the generalization ability of the frozen model can be leveraged to improve tool selection. We present Tool selECTion via meta-reasONing (TECTON), a two-phase system that first reasons over a task using a custom fine-tuned LM head and outputs candidate tools. Then, with the custom head disabled, it meta-reasons (i.e., it reasons over the previous reasoning process) to make a final choice. We show that TECTON results in substantial gains - both in-distribution and out-of-distribution - on a range of math reasoning datasets.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively utilize external tools in large - scale language models (LLMs) to improve their performance in knowledge - intensive tasks, especially in mathematical reasoning tasks. Existing methods either learn tool use through context examples or perform full - model fine - tuning on labeled data. However, these methods all have scalability problems and it is difficult to handle a large number of tools or adapt to new tool sets. To solve these problems, the authors propose a new two - stage framework - Tool selECTion via meta - reas ONing (TECTON), aiming to improve tool selection through meta - reasoning, thereby improving the performance of LLMs in mathematical reasoning tasks.

Specifically, the main contributions of the paper are as follows:
1. **Introduced TECTON**: A two - stage framework that combines a custom - fine - tuned head and a frozen LLM to improve tool use in mathematical reasoning tasks.
2. **Demonstrated the superior performance of TECTON**: TECTON outperforms strong baseline models both on in - distribution data and unseen benchmark tests.
3. **Enhanced three popular mathematical reasoning data sets**: Made them more challenging and shared the data and code.

Through these contributions, the paper aims to show how to utilize the generalization ability of LLMs to improve tool selection through meta - reasoning, so as to achieve better performance in complex tasks.