Abstract:Large Language Models (LLMs) have been successful in mathematical reasoning tasks such as formal theorem proving when integrated with interactive proof assistants like Lean. Existing approaches involve training or fine-tuning an LLM on a specific dataset to perform well on particular domains, such as undergraduate-level mathematics. These methods struggle with generalizability to advanced mathematics. A fundamental limitation is that these approaches operate on static domains, failing to capture how mathematicians often work across multiple domains and projects simultaneously or cyclically. We present LeanAgent, a novel lifelong learning framework for theorem proving that continuously generalizes to and improves on ever-expanding mathematical knowledge without forgetting previously learned knowledge. LeanAgent introduces several key innovations, including a curriculum learning strategy that optimizes the learning trajectory in terms of mathematical difficulty, a dynamic database for efficient management of evolving mathematical knowledge, and progressive training to balance stability and plasticity. LeanAgent successfully proves 162 theorems previously unproved by humans across 23 diverse Lean repositories, many from advanced mathematics. It performs significantly better than the static LLM baseline, proving challenging theorems in domains like abstract algebra and algebraic topology while showcasing a clear progression of learning from basic concepts to advanced topics. In addition, we analyze LeanAgent's superior performance on key lifelong learning metrics. LeanAgent achieves exceptional scores in stability and backward transfer, where learning new tasks improves performance on previously learned tasks. This emphasizes LeanAgent's continuous generalizability and improvement, explaining its superior theorem-proving performance.

Formalizing Mason-Stothers Theorem and its Corollaries in Lean 4

Mathematical Formalized Problem Solving and Theorem Proving in Different Fields in Lean 4

A complete formalization of Fermat's Last Theorem for regular primes in Lean

Formalizing Automated Market Makers in the Lean 4 Theorem Prover

Formalizing Geometric Algebra in Lean

Fermat's Last Theorem for regular primes

TheoremLlama: Transforming General-Purpose LLMs into Lean4 Experts

A Lean Dataset for International Math Olympiad: Small Steps towards Writing Math Proofs for Hard Problems

Certifying rings of integers in number fields

LeanReasoner: Boosting Complex Logical Reasoning with Lean

Lean4Lean: Towards a Verified Typechecker for Lean, in Lean

Formalization of physics index notation in Lean 4

Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs

Lean Workbook: A large-scale Lean problem set formalized from natural language math problems

LeanAgent: Lifelong Learning for Formal Theorem Proving

Investigations into Proof Structures

LeanDojo: Theorem Proving with Retrieval-Augmented Language Models

Formalized Haar Measure

An Evaluation Benchmark for Autoformalization in Lean4

Efficient Evaluations of Weighted Sums over the Boolean Lattice inspired by conjectures of Berti, Corsi, Maspero, and Ventura