How Do Humans Write Code? Large Models Do It the Same Way Too

Long Li,Xuzheng He,Haozhe Wang,Linlin Wang,Liang He

2024-10-16

Abstract:Program-of-Thought (PoT) replaces natural language-based Chain-of-Thought (CoT) as the most popular method in Large Language Models (LLMs) mathematical reasoning tasks by utilizing external tool calls to circumvent computational errors. However, our evaluation of the GPT-4 and Llama series reveals that using PoT introduces more reasoning errors, such as incorrect formulas or flawed logic, compared to CoT. To address this issue, we propose Human-Think Language (HTL), which leverages a suite of strategies that help integrate PoT and CoT, encompassing: (1) a new generation paradigm that uses full CoT reasoning to control code generation. (2) Focus Attention, that directs model attention to the CoT reasoning during PoT to generate more logical code. (3) reinforcement learning that utilizes the accuracy of both CoT and PoT responses as rewards to prevent repetitive reasoning steps in LLMs when solving difficult math problems. Our method achieves an average improvement of 6.5% on the Llama-Base model and 4.3% on the Mistral-Base model across 8 mathematical calculation datasets. It also shows significant effectiveness on five out-of-domain datasets by controlling the model's information flow, exhibiting strong transferability. Additionally, HTL shows the most significant improvement in non-mathematical natural language inference task, contributing to a unified reasoning task framework

Artificial Intelligence,Computation and Language,Programming Languages

What problem does this paper attempt to address?

The paper attempts to address the issues that large language models (LLMs) face when handling mathematical reasoning tasks, particularly the fact that the Program of Thought (PoT) approach is more prone to errors compared to the Chain of Thought (CoT) approach. Specifically, the paper points out:

1. **Calculation Errors**: Current large language models are prone to calculation errors when performing mathematical computations.
2. **Reasoning Errors**: When using the PoT approach, models are more likely to make reasoning errors or text comprehension errors when dealing with problems that are close to natural language scenarios, whereas these issues are almost non-existent when using the CoT approach.
3. **Code Translation Errors (CTE)**: Due to the significantly larger amount of natural language data compared to code data in the training set, the PoT approach is more likely to make errors when translating natural language into code.

To address these issues, the paper proposes a new method—**Human Thought Language (HTL)**, which aims to improve the accuracy and reliability of models in mathematical reasoning tasks by combining complete natural language reasoning (CoT) to control code generation (PoT).