DOP: Diagnostic-Oriented Prompting for Large Language Models in Mathematical Correction

Hao Chen,Biaojie Zeng,Xin Lin,Liang He,Aimin Zhou

2024-05-20

Abstract:Math world problems correction(MWPC) is a novel task dedicated to rectifying reasoning errors in the process of solving mathematical problems. In this paper, leveraging the advancements in large language models (LLMs), we address two key objectives:(1) Distinguishing between mathematical reasoning and error correction; (2) Exploring strategies to enhance the error correction capabilities of LLMs in mathematics to solve MWPC task. We noticed that, in real-time education,assisting students in recognizing their mistakes is more crucial than simply providing correct answers. However, current research tends to prioritize obtaining accurate solutions to math problems rather than correcting potentially incorrect ones. Therefore, we modify the research paradigm, demonstrating that improving mathematical reasoning abilities does not equate to mastery in error correction. Meanwhile, we propose a novel method called diagnostic-oriented promping(DOP) aimed at facilitating LLMs to excel in error correction. In experiments, DOP has shown outstanding performance, highlighting its significant impact. We argue that in mathematical education, the demand for outstanding correctors surpasses that for proficient reasoners. Codes and data are available on

Computation and Language

What problem does this paper attempt to address?

The paper attempts to address the issue of the disparity between large language models' (LLMs) mathematical reasoning and error correction capabilities in the process of solving mathematical problems. Specifically, the paper points out that current research mainly focuses on improving LLMs' ability to solve mathematical problems, while neglecting the importance of error correction. The authors observe that in real educational scenarios, helping students identify errors is more important than merely providing the correct answers. Therefore, the paper proposes the following two main objectives:

1. **Distinguish between mathematical reasoning and error correction**: The paper emphasizes that improving mathematical reasoning ability does not equate to mastering error correction ability. This means that even if a model can correctly solve a problem, it may still be ineffective at correcting errors.

2. **Explore strategies to enhance LLMs' error correction capabilities**: To achieve this goal, the paper introduces a new task—Mathematical Word Problem Correction (MWPC), and proposes a new method called Diagnostic-Oriented Prompting (DOP), aimed at leveraging modern educational resources to enhance LLMs' error correction capabilities.

Through experimental validation, the paper demonstrates the significant effect of the DOP method in improving LLMs' error correction capabilities, emphasizing that in mathematics education, an excellent error corrector is more important than a proficient reasoner.