Abstract:Despite their success in many natural language tasks, solving math problems remains a significant challenge for large language models (LLMs). A large gap exists between LLMs' pass-at-one and pass-at-N performance in solving math problems, suggesting LLMs might be close to finding correct solutions, motivating our exploration of fine-tuning methods to unlock LLMs' performance. Using the challenging MATH dataset, we investigate three fine-tuning strategies: (1) solution fine-tuning, where we fine-tune to generate a detailed solution for a given math problem; (2) solution-cluster re-ranking, where the LLM is fine-tuned as a solution verifier/evaluator to choose among generated candidate solution clusters; (3) multi-task sequential fine-tuning, which integrates both solution generation and evaluation tasks together efficiently to enhance the LLM performance. With these methods, we present a thorough empirical study on a series of PaLM 2 models and find: (1) The quality and style of the step-by-step solutions used for fine-tuning can make a significant impact on the model performance; (2) While solution re-ranking and majority voting are both effective for improving the model performance when used separately, they can also be used together for an even greater performance boost; (3) Multi-task fine-tuning that sequentially separates the solution generation and evaluation tasks can offer improved performance compared with the solution fine-tuning baseline. Guided by these insights, we design a fine-tuning recipe that yields approximately 58.8% accuracy on the MATH dataset with fine-tuned PaLM 2-L models, an 11.2% accuracy improvement over the few-shot performance of pre-trained PaLM 2-L model with majority voting.

Improving Math Problem Solving in Large Language Models Through Categorization and Strategy Tailoring

Large Language Models for Mathematical Reasoning: Progresses and Challenges

ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline

Large Language Models Are Unconscious of Unreasonability in Math Problems

MathLearner: A Large Language Model Agent Framework for Learning to Solve Mathematical Problems

Solving Math Word Problems by Combining Language Models With Symbolic Solvers

Do Large Language Models Truly Grasp Mathematics? An Empirical Exploration From A Psychological Perspective

Boosting Large Language Models with Socratic Method for Conversational Mathematics Teaching

Do Large Language Models Truly Grasp Mathematics? An Empirical Exploration From Cognitive Psychology

MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data

Improving Large Language Model Fine-tuning for Solving Math Problems

Novice Learner and Expert Tutor: Evaluating Math Reasoning Abilities of Large Language Models with Misconceptions

SIaM: Self-Improving Code-Assisted Mathematical Reasoning of Large Language Models

INC-Math: Integrating Natural Language and Code for Enhanced Mathematical Reasoning in Large Language Models

LLM Reasoning Engine: Specialized Training for Enhanced Mathematical Reasoning

Mathify: Evaluating Large Language Models on Mathematical Problem Solving Tasks

Large Language Models for Mathematical Analysis

Logic Contrastive Reasoning with Lightweight Large Language Model for Math Word Problems