Abstract:Large language models (LLMs) are known to struggle with complicated reasoning tasks such as math word problems (MWPs). In this paper, we present how analogy from similarly structured questions can improve LLMs' problem-solving capabilities for MWPs. Specifically, we rely on the retrieval of problems with similar computational graphs to the given question to serve as exemplars in the prompt, providing the correct reasoning path for the generation model to refer to. Empirical results across six math word problem datasets demonstrate the effectiveness of our proposed method, which achieves a significant improvement of up to 6.7 percent on average in absolute value, compared to baseline methods. These results highlight our method's potential in addressing the reasoning challenges in current LLMs.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: large - language models (LLMs) have difficulties in solving math word problems (MWPs). Although LLMs perform well on many tasks, they do not perform well on tasks that require complex reasoning, especially math word problems. These problems require not only language - understanding ability but also mathematical - reasoning ability, which poses a challenge to LLMs. The paper proposes a retrieval method based on computational graphs to enhance the ability of LLMs to solve math word problems by analogizing structurally similar problems. Specifically, the main contributions of the paper include: 1. **Proposing a retrieval method based on computational graphs**: This method can select examples with similar structures to the target problem, thereby providing more relevant problem - solving guidance in few - shot prompting. 2. **Training a structural - similarity retriever**: Developed a retriever model trained with contrastive learning, which can recognize structural similarities in math word problems. This lightweight and modular retriever can be seamlessly integrated into the LLM's reasoning process without updating the LLM's parameters. 3. **Extensive experiments and analysis**: Comprehensive experiments were carried out on six math word - problem datasets, and the results show that this method is significantly better than semantic - based retrieval and random - selection baseline methods, with an average absolute value improvement of 6.7%. Through these contributions, the paper aims to improve the reasoning ability of LLMs in solving math word problems, thereby expanding their applications in education, automatic tutoring, and complex - reasoning tasks.

Learning by Analogy: Enhancing Few-Shot Prompting for Math Word Problem Solving with Computational Graph-Based Retrieval

Achieving >97 Better Solvers for Math Word Problems

Teaching-Inspired Integrated Prompting Framework: A Novel Approach for Enhancing Reasoning in Large Language Models

Look Before You Leap: Problem Elaboration Prompting Improves Mathematical Reasoning in Large Language Models

DOP: Diagnostic-Oriented Prompting for Large Language Models in Mathematical Correction

MathPrompter: Mathematical Reasoning using Large Language Models

Recall and Learn: A Memory-augmented Solver for Math Word Problems

Logic Contrastive Reasoning with Lightweight Large Language Model for Math Word Problems

Leveraging Training Data in Few-Shot Prompting for Numerical Reasoning

Deep Learning in Automatic Math Word Problem Solvers

Expanding Search Space with Diverse Prompting Agents: An Efficient Sampling Approach for LLM Mathematical Reasoning

Learning by Analogy: Diverse Questions Generation in Math Word Problem

Achieving >97% on GSM8K: Deeply Understanding the Problems Makes LLMs Better Solvers for Math Word Problems

Reason-and-Execute Prompting: Enhancing Multi-Modal Large Language Models for Solving Geometry Questions

Gap-Filling Prompting Enhances Code-Assisted Mathematical Reasoning

LLMs can Find Mathematical Reasoning Mistakes by Pedagogical Chain-of-Thought

Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models

Solving Math Word Problems by Combining Language Models With Symbolic Solvers

Active Prompting with Chain-of-Thought for Large Language Models

From Good to Great: Improving Math Reasoning with Tool-Augmented Interleaf Prompting