Multi-tool Integration Application for Math Reasoning Using Large Language Model

Zhihua Duan,Jialin Wang

2024-08-22

Abstract:Mathematical reasoning is an important research direction in the field of artificial intelligence. This article proposes a novel multi tool application framework for mathematical reasoning, aiming to achieve more comprehensive and accurate mathematical reasoning by utilizing the collaborative effect of large language models (LLMs) and multiple external tools. Firstly, use a Math Tool to perform basic mathematical calculations during the inference process through interaction with LLM. Secondly, Code Tool can generate code fragments that comply with syntax rules and execute them, providing support for complex mathematical problems. Then, through the iterative reasoning of the CoT Tool, the logical coherence and accuracy of mathematical reasoning are enhanced. Ultimately, by using self consistency tools to select the final answer based on different parameters, the consistency and reliability of reasoning are improved. Through the synergistic effect of these tools, the framework has achieved significant performance improvement in mathematical reasoning tasks. We conducted experiments on the NumGLUE Task 4 test set, which includes 220 mathematical reasoning fill in the blank questions. The experimental results showed that, based on Math Tool, Code Tool, and CoT Tool, in Task 4 task,our method achieved an accuracy of 89.09,compared with the GPT3+FewShot baseline, Few Shot+ERNIE-4.0+self consistency improved by 49.09%, and compared with fine-tuning the Fine tuning baseline, Few Shot+ERNIE-4.0+self consistency improved by 52.29%

Artificial Intelligence

What problem does this paper attempt to address?

The paper attempts to address the problem of how to achieve more comprehensive and accurate mathematical reasoning in mathematical reasoning tasks by leveraging the synergy of large language models (LLMs) with multiple external tools. Specifically, the paper proposes a multi-tool application framework aimed at enhancing mathematical reasoning capabilities through the collaboration of the following tools:

1. **Math Tool**: Used to perform basic mathematical calculations during the reasoning process.
2. **Code Tool**: Generates syntactically correct code snippets and executes them to support complex mathematical problems.
3. **CoT Tool**: Enhances the logical coherence and accuracy of mathematical reasoning through iterative reasoning via Chain of Thought (CoT).
4. **Self Consistency Tool**: Improves the consistency and reliability of reasoning by selecting the final answer through a self-consistency tool.

The paper validates the effectiveness of this framework through experiments, particularly on the NumGLUE Task 4 test set, which contains 220 mathematical reasoning fill-in-the-blank questions. The experimental results show that, compared to baseline models, the proposed method achieved an accuracy of 89.09% on Task 4, which is 49.09% higher than the GPT-3+FewShot baseline and 52.29% higher than the fine-tuned baseline. This indicates that the synergy of multiple tools can significantly enhance the performance of mathematical reasoning tasks.