DeepMind AI solves geometry problems at star-student level

Davide Castelvecchi

DOI: https://doi.org/10.1038/d41586-024-00141-5

IF: 64.8

2024-01-18

Nature

Abstract:Algorithms are now as good at geometry as some of the world's most mathematically talented school kids. Algorithms are now as good at geometry as some of the world's most mathematically talented school kids.

multidisciplinary sciences

What problem does this paper attempt to address?

AI cracks hard maths

Alex Wilkins

DOI: https://doi.org/10.1016/s0262-4079(24)00157-x

IF: 0.906

2024-01-28

The New Scientist

Abstract:An AI from Google DeepMind scores almost as well as the best students on geometry questions from the International Mathematical Olympiad, finds Alex Wilkins

multidisciplinary sciences
DeepMind AI outdoes human mathematicians on unsolved problem

Davide Castelvecchi

DOI: https://doi.org/10.1038/d41586-023-04043-w

IF: 64.8

2023-12-15

Nature

Abstract:Large language model improves on efforts to solve combinatorics problems inspired by the card game Set. Large language model improves on efforts to solve combinatorics problems inspired by the card game Set.

multidisciplinary sciences
DeepMind AI creates algorithms that sort data faster than those built by people

Matthew Hutson

DOI: https://doi.org/10.1038/d41586-023-01883-4

IF: 64.8

2023-06-08

Nature

Abstract:The technology developed by DeepMind that plays Go and chess can also help to write code. The technology developed by DeepMind that plays Go and chess can also help to write code.

multidisciplinary sciences
DeepMind hits milestone in solving maths problems — AI's next grand challenge

Davide Castelvecchi

DOI: https://doi.org/10.1038/d41586-024-02441-2

IF: 64.8

2024-07-26

Nature

Abstract:AlphaProof showed its prowess on questions from this year's Mathematical Olympiad — a step in the race to create substantial proofs with artificial intelligence. AlphaProof showed its prowess on questions from this year's Mathematical Olympiad — a step in the race to create substantial proofs with artificial intelligence.

multidisciplinary sciences
A Deep Reinforcement Learning Agent for Geometry Online Tutoring

Ziyang Xiao,Dongxiang Zhang

DOI: https://doi.org/10.1007/s10115-022-01804-3

IF: 2.7

2022-01-01

Knowledge and Information Systems

Abstract:In this paper, we apply deep reinforcement learning (DRL) for geometry reasoning and develop Dragon to facilitate online tutoring. Its success is contingent on a flexible data model to capture diverse concepts and heterogeneous relations, as well as an effective DRL agent to generate near-optimal and human-readable solutions. We use proximal policy optimization (PPO) as the backbone DRL architecture, customized with effective state representation and integrated with a bunch of optimization tricks including attention mechanism, action mask, data augmentation and curriculum learning. In our experimental study, we craft so far the largest scale dataset with geometry problems and a knowledge base with 46 theorems. We implement various heuristic algorithms and DRL models as baselines for performance comparison. The results show that our agent achieves near-optimal solution and is superior over multiple competitive baselines. To benefit the community, we opensource the dataset and implementation at https://github.com/AIEdu-xzy/geometry-solver .
Solving olympiad geometry without human demonstrations

Trieu H. Trinh,Yuhuai Wu,Quoc V. Le,He He,Thang Luong

DOI: https://doi.org/10.1038/s41586-023-06747-5

IF: 64.8

2024-01-17

Nature

Abstract:Abstract Proving mathematical theorems at the olympiad level represents a notable milestone in human-level automated reasoning 1–4 , owing to their reputed difficulty among the world’s best talents in pre-university mathematics. Current machine-learning approaches, however, are not applicable to most mathematical domains owing to the high cost of translating human proofs into machine-verifiable format. The problem is even worse for geometry because of its unique translation challenges 1,5 , resulting in severe scarcity of training data. We propose AlphaGeometry, a theorem prover for Euclidean plane geometry that sidesteps the need for human demonstrations by synthesizing millions of theorems and proofs across different levels of complexity. AlphaGeometry is a neuro-symbolic system that uses a neural language model, trained from scratch on our large-scale synthetic data, to guide a symbolic deduction engine through infinite branching points in challenging problems. On a test set of 30 latest olympiad-level problems, AlphaGeometry solves 25, outperforming the previous best method that only solves ten problems and approaching the performance of an average International Mathematical Olympiad (IMO) gold medallist. Notably, AlphaGeometry produces human-readable proofs, solves all geometry problems in the IMO 2000 and 2015 under human expert evaluation and discovers a generalized version of a translated IMO theorem in 2004.

multidisciplinary sciences
Limits of an AI program for solving college math problems

Ernest Davis

DOI: https://doi.org/10.48550/arXiv.2208.06906

IF: 14.4

2022-08-14

Artificial Intelligence

Abstract:Drori et al. (2022) report that "A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level ... [It] automatically answers 81\% of university-level mathematics problems." The system they describe is indeed impressive; however, the above description is very much overstated. The work of solving the problems is done, not by a neural network, but by the symbolic algebra package Sympy. Problems of various formats are excluded from consideration. The so-called "explanations" are just rewordings of lines of code. Answers are marked as correct that are not in the form specified in the problem. Most seriously, it seems that in many cases the system uses the correct answer given in the test corpus to guide its path to solving the problem.
DeepMind AI is a games master

Matthew Sparkes

DOI: https://doi.org/10.1016/s0262-4079(23)02185-1

2023-11-01

Abstract:System that beats humans at chess and poker could be a step towards general intelligence

multidisciplinary sciences
Deep Learning in Automatic Math Word Problem Solvers

Dongxiang Zhang

DOI: https://doi.org/10.1007/978-3-031-09687-7_14

2022-01-01

Abstract:The design of an automatic solver for mathematical word problems (MWPs) dates back to the early 1960s and regained booming attention in recent years, owing to revolutionary advances in deep learning. Its objective is to parse the human-readable word problems into machine-understandable logical expressions. The problem is challenging due to the existence of a substantial semantic gap. To a certain extent, MWPs have been recognized as good test beds to evaluate the intelligence level of agents in terms of natural language understanding and automatic reasoning. The successful solving of MWPs can benefit online tutoring and constitute a milestone toward general AI. In this chapter, we present a general introduction to the technical evolution trend for MWP solvers in recent decades and pay particular attention to recent advancement with deep learning models. We also report their performances on public benchmark datasets, which can update readers’ understandings of the latest status of automatic math problem solvers.
A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Level

Iddo Drori,Sarah Zhang,Reece Shuttleworth,Leonard Tang,Albert Lu,Elizabeth Ke,Kevin Liu,Linda Chen,Sunny Tran,Newman Cheng,Roman Wang,Nikhil Singh,Taylor L. Patti,Jayson Lynch,Avi Shporer,Nakul Verma,Eugene Wu,Gilbert Strang

DOI: https://doi.org/10.1073/pnas.2123433119

2022-05-31

Abstract:We demonstrate that a neural network pre-trained on text and fine-tuned on code solves mathematics course problems, explains solutions, and generates new questions at a human level. We automatically synthesize programs using few-shot learning and OpenAI's Codex transformer and execute them to solve course problems at 81% automatic accuracy. We curate a new dataset of questions from MIT's largest mathematics courses (Single Variable and Multivariable Calculus, Differential Equations, Introduction to Probability and Statistics, Linear Algebra, and Mathematics for Computer Science) and Columbia University's Computational Linear Algebra. We solve questions from a MATH dataset (on Prealgebra, Algebra, Counting and Probability, Intermediate Algebra, Number Theory, and Precalculus), the latest benchmark of advanced mathematics problems designed to assess mathematical reasoning. We randomly sample questions and generate solutions with multiple modalities, including numbers, equations, and plots. The latest GPT-3 language model pre-trained on text automatically solves only 18.8% of these university questions using zero-shot learning and 30.8% using few-shot learning and the most recent chain of thought prompting. In contrast, program synthesis with few-shot learning using Codex fine-tuned on code generates programs that automatically solve 81% of these questions. Our approach improves the previous state-of-the-art automatic solution accuracy on the benchmark topics from 8.8% to 81.1%. We perform a survey to evaluate the quality and difficulty of generated questions. This work is the first to automatically solve university-level mathematics course questions at a human level and the first work to explain and generate university-level mathematics course questions at scale, a milestone for higher education.

Machine Learning,Artificial Intelligence
Google AI algorithm masters ancient game of Go

Elizabeth Gibney

DOI: https://doi.org/10.1038/529445a

IF: 64.8

2016-01-27

Nature

Abstract:Deep-learning software defeats human professional for first time.

multidisciplinary sciences
Incorporating Graph Attention Mechanism into Geometric Problem Solving Based on Deep Reinforcement Learning

Xiuqin Zhong,Shengyuan Yan,Gongqi Lin,Hongguang Fu,Liang Xu,Siwen Jiang,Lei Huang,Wei Fang

2024-03-14

Abstract:In the context of online education, designing an automatic solver for geometric problems has been considered a crucial step towards general math Artificial Intelligence (AI), empowered by natural language understanding and traditional logical inference. In most instances, problems are addressed by adding auxiliary components such as lines or points. However, adding auxiliary components automatically is challenging due to the complexity in selecting suitable auxiliary components especially when pivotal decisions have to be made. The state-of-the-art performance has been achieved by exhausting all possible strategies from the category library to identify the one with the maximum likelihood. However, an extensive strategy search have to be applied to trade accuracy for ef-ficiency. To add auxiliary components automatically and efficiently, we present deep reinforcement learning framework based on the language model, such as BERT. We firstly apply the graph attention mechanism to reduce the strategy searching space, called AttnStrategy, which only focus on the conclusion-related components. Meanwhile, a novel algorithm, named Automatically Adding Auxiliary Components using Reinforcement Learning framework (A3C-RL), is proposed by forcing an agent to select top strategies, which incorporates the AttnStrategy and BERT as the memory components. Results from extensive experiments show that the proposed A3C-RL algorithm can substantially enhance the average precision by 32.7% compared to the traditional MCTS. In addition, the A3C-RL algorithm outperforms humans on the geometric questions from the annual University Entrance Mathematical Examination of China.

Computers and Society,Artificial Intelligence,Computation and Language,Machine Learning
MathDQN: Solving Arithmetic Word Problems Via Deep Reinforcement Learning.

Lei Wang,Dongxiang Zhang,Lianli Gao,Jingkuan Song,Long Guo,Heng Tao Shen

DOI: https://doi.org/10.1609/aaai.v32i1.11981

2018-01-01

Proceedings of the AAAI Conference on Artificial Intelligence

Abstract:Designing an automatic solver for math word problems has been considered as a crucial step towards general AI, with the ability of natural language understanding and logical inference. The state-of-the-art performance was achieved by enumerating all the possible expressions from the quantities in the text and customizing a scoring function to identify the one with the maximum probability. However, it incurs exponential search space with the number of quantities and beam search has to be applied to trade accuracy for efficiency. In this paper, we make the first attempt of applying deep reinforcement learning to solve arithmetic word problems. The motivation is that deep Q-network has witnessed success in solving various problems with big search space and achieves promising performance in terms of both accuracy and running time. To fit the math problem scenario, we propose our MathDQN that is customized from the general deep reinforcement learning framework. Technically, we design the states, actions, reward function, together with a feed-forward neural network as the deep Q-network. Extensive experimental results validate our superiority over state-of-the-art methods. Our MathDQN yields remarkable improvement on most of datasets and boosts the average precision among all the benchmark datasets by 15\%.
Advancing mathematics by guiding human intuition with AI

Alex Davies,Petar Veličković,Lars Buesing,Sam Blackwell,Daniel Zheng,Nenad Tomašev,Richard Tanburn,Peter Battaglia,Charles Blundell,András Juhász,Marc Lackenby,Geordie Williamson,Demis Hassabis,Pushmeet Kohli

DOI: https://doi.org/10.1038/s41586-021-04086-x

IF: 64.8

2021-12-01

Nature

Abstract:Abstract The practice of mathematics involves discovering patterns and using these to formulate and prove conjectures, resulting in theorems. Since the 1960s, mathematicians have used computers to assist in the discovery of patterns and formulation of conjectures 1 , most famously in the Birch and Swinnerton-Dyer conjecture 2 , a Millennium Prize Problem 3 . Here we provide examples of new fundamental results in pure mathematics that have been discovered with the assistance of machine learning—demonstrating a method by which machine learning can aid mathematicians in discovering new conjectures and theorems. We propose a process of using machine learning to discover potential patterns and relations between mathematical objects, understanding them with attribution techniques and using these observations to guide intuition and propose conjectures. We outline this machine-learning-guided framework and demonstrate its successful application to current research questions in distinct areas of pure mathematics, in each case showing how it led to meaningful mathematical contributions on important open problems: a new connection between the algebraic and geometric structure of knots, and a candidate algorithm predicted by the combinatorial invariance conjecture for symmetric groups 4 . Our work may serve as a model for collaboration between the fields of mathematics and artificial intelligence (AI) that can achieve surprising results by leveraging the respective strengths of mathematicians and machine learning.

multidisciplinary sciences
Wu's Method can Boost Symbolic AI to Rival Silver Medalists and AlphaGeometry to Outperform Gold Medalists at IMO Geometry

Shiven Sinha,Ameya Prabhu,Ponnurangam Kumaraguru,Siddharth Bhat,Matthias Bethge

2024-04-11

Abstract:Proving geometric theorems constitutes a hallmark of visual reasoning combining both intuitive and logical skills. Therefore, automated theorem proving of Olympiad-level geometry problems is considered a notable milestone in human-level automated reasoning. The introduction of AlphaGeometry, a neuro-symbolic model trained with 100 million synthetic samples, marked a major breakthrough. It solved 25 of 30 International Mathematical Olympiad (IMO) problems whereas the reported baseline based on Wu's method solved only ten. In this note, we revisit the IMO-AG-30 Challenge introduced with AlphaGeometry, and find that Wu's method is surprisingly strong. Wu's method alone can solve 15 problems, and some of them are not solved by any of the other methods. This leads to two key findings: (i) Combining Wu's method with the classic synthetic methods of deductive databases and angle, ratio, and distance chasing solves 21 out of 30 methods by just using a CPU-only laptop with a time limit of 5 minutes per problem. Essentially, this classic method solves just 4 problems less than AlphaGeometry and establishes the first fully symbolic baseline strong enough to rival the performance of an IMO silver medalist. (ii) Wu's method even solves 2 of the 5 problems that AlphaGeometry failed to solve. Thus, by combining AlphaGeometry with Wu's method we set a new state-of-the-art for automated theorem proving on IMO-AG-30, solving 27 out of 30 problems, the first AI method which outperforms an IMO gold medalist.

Artificial Intelligence,Computational Geometry,Computation and Language,Machine Learning
‘It will change everything’: DeepMind’s AI makes gigantic leap in solving protein structures

Ewen Callaway

DOI: https://doi.org/10.1038/d41586-020-03348-4

IF: 64.8

2020-11-30

Nature

Abstract:An artificial intelligence (AI) network developed by Google AI offshoot DeepMind has made a gargantuan leap in solving one of biology's grandest challenges — determining a protein's 3D shape from its amino-acid sequence.

multidisciplinary sciences
FGeo-DRL: Deductive Reasoning for Geometric Problems through Deep Reinforcement Learning

Jia Zou,Xiaokai Zhang,Yiming He,Na Zhu,Tuo Leng

DOI: https://doi.org/10.3390/sym16040437

2024-04-06

Symmetry

Abstract:Human-like automatic deductive reasoning has always been one of the most challenging open problems in the interdisciplinary field of mathematics and artificial intelligence. This paper is the third in a series of our works. We built a neural-symbolic system, named FGeo-DRL, to automatically perform human-like geometric deductive reasoning. The neural part is an AI agent based on deep reinforcement learning, capable of autonomously learning problem-solving methods from the feedback of a formalized environment, without the need for human supervision. It leverages a pre-trained natural language model to establish a policy network for theorem selection and employ Monte Carlo Tree Search for heuristic exploration. The symbolic part is a reinforcement learning environment based on geometry formalization theory and FormalGeo, which models geometric problem solving (GPS) as a Markov Decision Process (MDP). In the formal symbolic system, the symmetry of plane geometric transformations ensures the uniqueness of geometric problems when converted into states. Finally, the known conditions and objectives of the problem form the state space, while the set of theorems forms the action space. Leveraging FGeo-DRL, we have achieved readable and verifiable automated solutions to geometric problems. Experiments conducted on the formalgeo7k dataset have achieved a problem-solving success rate of 86.40%.

multidisciplinary sciences
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges

Michael M. Bronstein,Joan Bruna,Taco Cohen,Petar Veličković

2021-05-03

Abstract:The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods. Indeed, many high-dimensional learning tasks previously thought to be beyond reach -- such as computer vision, playing Go, or protein folding -- are in fact feasible with appropriate computational scale. Remarkably, the essence of deep learning is built from two simple algorithmic principles: first, the notion of representation or feature learning, whereby adapted, often hierarchical, features capture the appropriate notion of regularity for each task, and second, learning by local gradient-descent type methods, typically implemented as backpropagation. While learning generic functions in high dimensions is a cursed estimation problem, most tasks of interest are not generic, and come with essential pre-defined regularities arising from the underlying low-dimensionality and structure of the physical world. This text is concerned with exposing these regularities through unified geometric principles that can be applied throughout a wide spectrum of applications. Such a 'geometric unification' endeavour, in the spirit of Felix Klein's Erlangen Program, serves a dual purpose: on one hand, it provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNNs, GNNs, and Transformers. On the other hand, it gives a constructive procedure to incorporate prior physical knowledge into neural architectures and provide principled way to build future architectures yet to be invented.

Machine Learning,Artificial Intelligence,Computational Geometry,Computer Vision and Pattern Recognition
GAPS: Geometry-Aware Problem Solver

Jiaxin Zhang,Yinghui Jiang,Yashar Moshfeghi

2024-01-30

Abstract:Geometry problem solving presents a formidable challenge within the NLP community. Existing approaches often rely on models designed for solving math word problems, neglecting the unique characteristics of geometry math problems. Additionally, the current research predominantly focuses on geometry calculation problems, while overlooking other essential aspects like proving. In this study, we address these limitations by proposing the Geometry-Aware Problem Solver (GAPS) model. GAPS is specifically designed to generate solution programs for geometry math problems of various types with the help of its unique problem-type classifier. To achieve this, GAPS treats the solution program as a composition of operators and operands, segregating their generation processes. Furthermore, we introduce the geometry elements enhancement method, which enhances the ability of GAPS to recognize geometry elements accurately. By leveraging these improvements, GAPS showcases remarkable performance in resolving geometry math problems. Our experiments conducted on the UniGeo dataset demonstrate the superiority of GAPS over the state-of-the-art model, Geoformer. Specifically, GAPS achieves an accuracy improvement of more than 5.3% for calculation tasks and an impressive 41.1% for proving tasks. Notably, GAPS achieves an impressive accuracy of 97.5% on proving problems, representing a significant advancement in solving geometry proving tasks.

Artificial Intelligence,Computation and Language

DeepMind AI solves geometry problems at star-student level

AI cracks hard maths

DeepMind AI outdoes human mathematicians on unsolved problem

DeepMind AI creates algorithms that sort data faster than those built by people

DeepMind hits milestone in solving maths problems — AI's next grand challenge

A Deep Reinforcement Learning Agent for Geometry Online Tutoring

Solving olympiad geometry without human demonstrations

Limits of an AI program for solving college math problems

DeepMind AI is a games master

Deep Learning in Automatic Math Word Problem Solvers

A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Level

Google AI algorithm masters ancient game of Go

Incorporating Graph Attention Mechanism into Geometric Problem Solving Based on Deep Reinforcement Learning

MathDQN: Solving Arithmetic Word Problems Via Deep Reinforcement Learning.

Advancing mathematics by guiding human intuition with AI

Wu's Method can Boost Symbolic AI to Rival Silver Medalists and AlphaGeometry to Outperform Gold Medalists at IMO Geometry

‘It will change everything’: DeepMind’s AI makes gigantic leap in solving protein structures

FGeo-DRL: Deductive Reasoning for Geometric Problems through Deep Reinforcement Learning

Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges

GAPS: Geometry-Aware Problem Solver