Abstract:Large Language Models (LLMs) demonstrate ever-increasing abilities in mathematical and algorithmic tasks, yet their geometric reasoning skills are underexplored. We investigate LLMs' abilities in constructive geometric problem-solving one of the most fundamental steps in the development of human mathematical reasoning. Our work reveals notable challenges that the state-of-the-art LLMs face in this domain despite many successes in similar areas. LLMs exhibit biases in target variable selection and struggle with 2D spatial relationships, often misrepresenting and hallucinating objects and their placements. To this end, we introduce a framework that formulates an LLMs-based multi-agents system that enhances their existing reasoning potential by conducting an internal dialogue. This work underscores LLMs' current limitations in geometric reasoning and improves geometric reasoning capabilities through self-correction, collaboration, and diverse role specializations.

What problem does this paper attempt to address?

This paper attempts to address the problems existing in large - language models (LLMs) in geometric reasoning. Although LLMs perform well in mathematical and algorithmic tasks, they still face significant challenges in constructive geometric problem - solving. Specifically: 1. **Insufficient geometric reasoning ability**: LLMs show obvious difficulties in dealing with 2D spatial relationships, often mis - representing or "hallucinating" objects and their positions. This lack of ability limits their performance in tasks requiring tool use, planning, and spatial reasoning. 2. **Variable selection bias**: LLMs have biases in selecting target variables and are easily influenced by variable names, leading to unnecessary complexity and incorrect solutions. 3. **Instruction - following problems**: LLMs tend to imitate the style in examples rather than focus on reasoning and appropriate abstraction, which makes them rely on specific patterns when solving geometric problems and ignore the logic and steps of the problems themselves. To solve these problems, the author proposes a framework to enhance the geometric reasoning ability of LLMs through the following methods: - **Multi - round dialogue system**: Introduce multiple agents based on LLMs, each agent undertakes a specific role and task, and collaboratively solve problems through internal dialogue. - **Adaptive prompting mechanism**: Dynamically adjust the prompt content so that LLMs can learn from previous interactions and avoid the limitations of static prompts. - **Variable renaming technique**: Rename the target variables to more general symbols to reduce the bias caused by variable names. - **Visual - relationship prompting**: Utilize visual - language large models (VLLM) to assist in understanding the spatial relationships in geometric scenes and improve the understanding and problem - solving ability of geometric problems. Through these improvements, the paper aims to bridge the ability gap of LLMs in geometric reasoning and improve their performance in this field.

Beyond Lines and Circles: Unveiling the Geometric Reasoning Gap in Large Language Models

Reasoning in Large Language Models: A Geometric Perspective

GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoning

G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model

Large Language Models for Mathematical Reasoning: Progresses and Challenges

EAGLE: Elevating Geometric Reasoning through LLM-empowered Visual Instruction Tuning

Do Large Language Models Truly Grasp Mathematics? An Empirical Exploration From A Psychological Perspective

Can Large Language Models Understand Symbolic Graphics Programs?

Can Large Language Models Create New Knowledge for Spatial Reasoning Tasks?

LLM Reasoning Engine: Specialized Training for Enhanced Mathematical Reasoning

Exploring Group and Symmetry Principles in Large Language Models

Do Large Language Models Truly Grasp Mathematics? An Empirical Exploration From Cognitive Psychology

The Curious Case of Nonverbal Abstract Reasoning with Multi-Modal Large Language Models

Reason-and-Execute Prompting: Enhancing Multi-Modal Large Language Models for Solving Geometry Questions

Geo-LLaVA: A Large Multi-Modal Model for Solving Geometry Math Problems with Meta In-Context Learning

Are Large-Language Models Graph Algorithmic Reasoners?

Improving Multimodal LLMs Ability In Geometry Problem Solving, Reasoning, And Multistep Scoring

A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges

Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generation

GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models