Abstract:With the boom of Large Language Models (LLMs), the research of solving Math Word Problem (MWP) has recently made great progress. However, there are few studies to examine the security of LLMs in math solving ability. Instead of attacking prompts in the use of LLMs, we propose a MathAttack model to attack MWP samples which are closer to the essence of security in solving math problems. Compared to traditional text adversarial attack, it is essential to preserve the mathematical logic of original MWPs during the attacking. To this end, we propose logical entity recognition to identify logical entries which are then frozen. Subsequently, the remaining text are attacked by adopting a word-level attacker. Furthermore, we propose a new dataset RobustMath to evaluate the robustness of LLMs in math solving ability. Extensive experiments on our RobustMath and two another math benchmark datasets GSM8K and MultiAirth show that MathAttack could effectively attack the math solving ability of LLMs. In the experiments, we observe that (1) Our adversarial samples from higher-accuracy LLMs are also effective for attacking LLMs with lower accuracy (e.g., transfer from larger to smaller-size LLMs, or from few-shot to zero-shot prompts); (2) Complex MWPs (such as more solving steps, longer text, more numbers) are more vulnerable to attack; (3) We can improve the robustness of LLMs by using our adversarial samples in few-shot prompts. Finally, we hope our practice and observation can serve as an important attempt towards enhancing the robustness of LLMs in math solving ability. We will release our code and dataset.

What problem does this paper attempt to address?

The paper primarily focuses on addressing the security and robustness issues of large language models (LLMs) when solving Math Word Problems (MWPs). Specifically: 1. **Research Background**: With the development of large language models, significant progress has been made in solving math word problems. However, there is currently little research focusing on the security and robustness of these models when solving mathematical problems. 2. **Core Issue**: The paper aims to evaluate the security of LLMs in solving mathematical problems by attacking the MWP samples themselves, rather than just attacking the prompts. This method of directly attacking the problem samples can more accurately reflect the security vulnerabilities of the models when handling mathematical tasks. 3. **Method Introduction**: A new method called MathAttack is proposed. This method first identifies and freezes the logical entities in the math word problems (such as characters, numbers, and scenarios), and then uses lexical-level attack techniques to modify the non-logical entities, thereby generating adversarial samples that retain the original mathematical logic but can mislead the model. 4. **Dataset Contribution**: To evaluate the effectiveness of MathAttack, the authors created a new dataset named RobustMath, which contains 300 high-quality adversarial samples of MWPs. 5. **Experimental Results**: Multiple benchmark tests show that MathAttack can effectively reduce the performance of LLMs of different scales in solving math word problems, and it is found that more complex math word problems are more susceptible to attacks. Additionally, using adversarial samples as few-shot prompts can improve the robustness of the model without sacrificing its problem-solving ability. In summary, the paper attempts to reveal and enhance the security and robustness of LLMs in solving math word problems by developing a new attack framework, MathAttack.

MathAttack: Attacking Large Language Models Towards Math Solving Ability

MathAttack: Attacking Large Language Models towards Math Solving Ability

LLM-Resistant Math Word Problem Generation via Adversarial Attacks

Investigating the Robustness of LLMs on Math Word Problems

Adversarial Math Word Problem Generation

MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solvers

MWPToolkit: an Open-Source Framework for Deep Learning-Based Math Word Problem Solvers.

Cutting Through the Noise: Boosting LLM Performance on Math Word Problems

What Makes Math Word Problems Challenging for LLMs?

MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data

Can LLMs Solve longer Math Word Problems Better?

MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models

GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers

Investigating Math Word Problems using Pretrained Multilingual Language Models

Mathify: Evaluating Large Language Models on Mathematical Problem Solving Tasks

Exposing the Achilles' Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical Reasoning

Large Language Models for Mathematical Analysis

Can Language Models Rival Mathematics Students? Evaluating Mathematical Reasoning through Textual Manipulation and Human Experiments

Target-driven Attack for Large Language Models

Solving Math Word Problems by Combining Language Models With Symbolic Solvers

Assessing Adversarial Robustness of Large Language Models: An Empirical Study