Abstract:With the introduction of large language models (LLMs), automatic math reasoning has seen tremendous success. However, current methods primarily focus on providing solutions or using techniques like Chain-of-Thought to enhance problem-solving accuracy. In this paper, we focus on improving the capability of mathematics teaching via a Socratic teaching-based LLM (\texttt{SocraticLLM}), which guides learners toward profound thinking with clarity and self-discovery via conversation. We collect and release a high-quality mathematical teaching dataset, named \texttt{SocraticMATH}, which provides Socratic-style conversations of problems with extra knowledge. Also, we propose a knowledge-enhanced LLM as a strong baseline to generate reliable responses with review, guidance/heuristic, rectification, and summarization. Experimental results show the great advantages of \texttt{SocraticLLM} by comparing it with several strong generative models. The codes and datasets are available on \url{<a class="link-external link-https" href="https://github.com/ECNU-ICALK/SocraticMath" rel="external noopener nofollow">this https URL</a>}.
What problem does this paper attempt to address?
The problem this paper attempts to address is how to enhance the effectiveness of mathematics teaching by combining the Socratic method with large language models (LLMs). Specifically, existing mathematical LLMs mainly focus on improving mathematical problem-solving abilities, but they have limitations in teaching applications. For example, when faced with mathematical problems, LLMs tend to directly provide the solution process, even when playing the role of a teacher. This leads to a lack of independent thinking and discovery process for students. Therefore, this study aims to design an LLM based on the Socratic method (referred to as SocraticLLM) to guide students to achieve deep thinking and self-discovery through dialogue, thereby improving the quality of mathematics teaching.
To achieve this goal, the researchers proposed several key innovations:
1. **Knowledge-enhanced SocraticLLM**: As a powerful baseline model, SocraticLLM generates reliable responses by incorporating additional knowledge, including teaching skills such as review, inspiration, correction, and summarization.
2. **Building a high-quality dataset**: The researchers created and released a high-quality mathematics teaching dataset named SocraticMATH, which includes original questions, answers, solutions, and dialogue tutoring data covering 513 knowledge points in elementary mathematics.
3. **Experimental validation**: Through a series of experiments, the researchers demonstrated the effectiveness of SocraticLLM in both automatic and manual evaluations, particularly excelling in reducing hallucinations (i.e., the generation of incorrect information by the model).
In summary, the core issue of this paper is to explore how to use the Socratic method to improve the application of LLMs in mathematics teaching, in order to promote students' independent learning and deep understanding.