Abstract:Math formulas (e.g., "distance = speed X time'') serve as one of the fundamental commonsense knowledge in human cognition, where humans naturally acquire and manipulate them in logical thinking for mathematical reasoning problems. However, existing reasoning models mainly focus on learning heuristic linguistics or patterns to generate answers, but do not pay enough attention on learning with such formula knowledge. Thus, they are not transparent (thus uninterpretable) in terms of understanding and grasping basic mathematical logic. In this paper, to promote a step forward in the domain, we first construct two datasets (Math23K-F and MAWPS-F) with precise annotations of formula usage in each reasoning step for math word problems. Especially, our datasets are refined on the benchmark datasets, and thus ensure the generality and comparability for relevant research. Then, we propose a novel Formula-mastered Solver (FOMAS) with the guidance of mastering formula knowledge to solve the problems. Specifically, we establish FOMAS with two systems drawing insight from the dual process theory, including a Knowledge System and a Reasoning System, to learn and apply formula knowledge, respectively. The Knowledge System accumulates the math formulas, where we propose a novel pretraining manner to mimic how humans grasp the mathematical logic behind them. Then, in the Reasoning System, we develop elaborate formula-guided symbol prediction and goal generation methods that retrieve the necessary formula knowledge from Knowledge System to improve both reasoning accuracy and interpretability. It organically simulates how humans conduct complex reasoning under the explicit instruction of math formulas. Experimental results prove that FOMAS has a stronger reasoning ability and achieves a more interpretable reasoning process, which verifies the necessity of introducing formula knowledge transparently.

FormulaReasoning: A Dataset for Formula-Based Numerical Reasoning

Guiding Mathematical Reasoning Via Mastering Commonsense Formula Knowledge

FinLLMs: A Framework for Financial Reasoning Dataset Generation with Large Language Models

Enhancing Numerical Reasoning with the Guidance of Reliable Reasoning Processes

P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains

When Do Program-of-Thought Works for Reasoning?

An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning

Towards Question Format Independent Numerical Reasoning: A Set of Prerequisite Tasks

LogicPro: Improving Complex Logical Reasoning via Program-Guided Learning

CHAMP: A Competition-level Dataset for Fine-Grained Analyses of LLMs' Mathematical Reasoning Capabilities

LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models

FinQA: A Dataset of Numerical Reasoning over Financial Data

ReasonAgain: Using Extractable Symbolic Programs to Evaluate Mathematical Reasoning

LLMs for Relational Reasoning: How Far are We?

FORTAP: Using Formulas for Numerical-Reasoning-Aware Table Pretraining

CBR-Ren: A Case-Based Reasoning Driven Retriever-Generator Model for Hybrid Long-Form Numerical Reasoning

Unleashing Reasoning Capability of LLMs via Scalable Question Synthesis from Scratch

FineMath: A Fine-Grained Mathematical Evaluation Benchmark for Chinese Large Language Models