Abstract:Consider the math problem: "Lily received 3 cookies from her best friend yesterday and ate 5 for breakfast. Today, her friend gave her 3 more cookies. How many cookies does Lily have now?" Many large language models (LLMs) in previous research approach this problem by calculating the answer "1" using the equation "3 - 5 + 3." However, from a human perspective, we recognize the inherent flaw in this problem: Lily cannot eat 5 cookies if she initially only had 3. This discrepancy prompts a key question: Are current LLMs merely Blind Solver that apply mathematical operations without deeper reasoning, or can they function as Logical Thinker capable of identifying logical inconsistencies? To explore this question, we propose a benchmark dataset, FaultyMath, which includes faulty math problems of rich diversity: i) multiple mathematical categories, e.g., algebra, geometry, number theory, etc., ii) varying levels of difficulty, and iii) different origins of faultiness -- ranging from violations of common sense and ambiguous statements to mathematical contradictions and more. We evaluate a broad spectrum of LLMs, including open-source, closed-source, and math-specialized models, using FaultyMath across three dimensions: (i) How accurately can the models detect faulty math problems without being explicitly prompted to do so? (ii) When provided with hints -- either correct or misleading -- about the validity of the problems, to what extent do LLMs adapt to become reliable Logical Thinker? (iii) How trustworthy are the explanations generated by LLMs when they recognize a math problem as flawed? Through extensive experimentation and detailed analysis, our results demonstrate that existing LLMs largely function as Blind Solver and fall short of the reasoning capabilities required to perform as Logical Thinker.

Don't Trust: Verify -- Grounding LLM Quantitative Reasoning with Autoformalization

Not All Votes Count! Programs as Verifiers Improve Self-Consistency of Language Models for Math Reasoning

VerityMath: Advancing Mathematical Reasoning by Self-Verification Through Unit Consistency

Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification

A Careful Examination of Large Language Model Performance on Grade School Arithmetic

Large Language Models Are Better Reasoners with Self-Verification

Autoformalize Mathematical Statements by Symbolic Equivalence and Semantic Consistency

LLM2: Let Large Language Models Harness System 2 Reasoning

Proof Automation with Large Language Models

From Blind Solvers to Logical Thinkers: Benchmarking LLMs' Logical Integrity on Faulty Mathematical Problems

Evaluating LLMs' Mathematical and Coding Competency through Ontology-guided Interventions

Large Language Models are reasoners with Self-Verification

Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correction

Exposing the Achilles' Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical Reasoning

Large Language Models Are Unconscious of Unreasonability in Math Problems

Evaluating LLMs' Mathematical Reasoning in Financial Document Question Answering

Novice Learner and Expert Tutor: Evaluating Math Reasoning Abilities of Large Language Models with Misconceptions

LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback

InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning