A Semantic Search Engine for Mathlib4

Guoxiong Gao,Haocheng Ju,Jiedong Jiang,Zihan Qin,Bin Dong

2024-03-20

Abstract:The interactive theorem prover, Lean, enables the verification of formal mathematical proofs and is backed by an expanding community. Central to this ecosystem is its mathematical library, mathlib4, which lays the groundwork for the formalization of an expanding range of mathematical theories. However, searching for theorems in mathlib4 can be challenging. To successfully search in mathlib4, users often need to be familiar with its naming conventions or documentation strings. Therefore, creating a semantic search engine that can be used easily by individuals with varying familiarity with mathlib4 is very important. In this paper, we present a semantic search engine for mathlib4 that accepts informal queries and finds the relevant theorems. We also establish a benchmark for assessing the performance of various search engines for mathlib4.

Information Retrieval,Machine Learning,Logic in Computer Science

What problem does this paper attempt to address?

This paper introduces a semantic search engine for the mathematical library mathlib4. mathlib4 is a mathematical library for the Lean interactive theorem prover, which contains a growing formalized foundation of mathematics. Currently, users face difficulties in finding theorems in mathlib4 because they need to be familiar with naming conventions or document strings. The paper proposes a search engine that accepts informal queries and finds relevant theorems, aiming to facilitate users with different levels of familiarity.

The paper first introduces two existing methods to search for mathlib4 theorems, namely document search and GitHub repository search. However, these methods have limitations, such as being difficult for beginners who are not familiar with naming conventions, and many theorems lacking document strings, leading to low search efficiency.

To address these issues, the paper proposes a semantic search engine that transforms the formalized statements of mathlib4 theorems into informal versions and builds a database containing these pairs. When users input informal queries, the system enhances query understanding and performs semantic search to find relevant results. In addition, the paper establishes a benchmark to evaluate the performance of different search engines on mathlib4.

The subsequent parts of the paper provide detailed descriptions of the methodology, including using large-scale language models for theorem informalization, building a semantic search engine, and query enhancement techniques. The experimental section compares the performance of different retrieval methods and analyzes the effects of task instructions, document content types, and query enhancement on the results.