Abstract:Analogical reasoning is a fundamental capacity of human cognition that allows us to reason abstractly about novel situations by relating them to past experiences. While it is thought to be essential for robust reasoning in AI systems, conventional approaches require significant training and/or hard-coding of domain knowledge to be applied to benchmark tasks. Inspired by cognitive science research that has found connections between human language and analogy-making, we explore the use of intuitive language-based abstractions to support analogy in AI systems. Specifically, we apply large pre-trained language models (PLMs) to visual Raven's Progressive Matrices (RPM), a common relational reasoning test. By simply encoding the perceptual features of the problem into language form, we find that PLMs exhibit a striking capacity for zero-shot relational reasoning, exceeding human performance and nearing supervised vision-based methods. We explore different encodings that vary the level of abstraction over task features, finding that higher-level abstractions further strengthen PLMs' analogical reasoning. Our detailed analysis reveals insights on the role of model complexity, in-context learning, and prior knowledge in solving RPM tasks.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to explore how to use pre - trained language models (PLMs) for analogical reasoning, especially under zero - shot conditions. Specifically, the researchers hope to answer the following core questions: 1. **Challenges of analogical reasoning in AI systems**: - Traditional analogical reasoning methods require a large amount of training data or hard - coded domain knowledge, which limits their generalization ability. - The researchers hope to simplify this process through natural language processing methods, enabling AI systems to perform effective analogical reasoning without extensive training. 2. **The role of language in analogical reasoning**: - Cognitive science research shows that human language and symbol systems play a crucial role in analogical reasoning. For example, language can help people abstract and better understand complex perceptual information. - The researchers explored whether language can play a similar role in AI systems to assist in analogical reasoning. 3. **Applying PLMs for visual analogical reasoning**: - The researchers selected a classic analogical reasoning test - Raven’s Progressive Matrices (RPM) and attempted to transform visual features into language descriptions to evaluate the analogical reasoning ability of PLMs under zero - shot conditions. - The RPM task requires inferring the missing part from a given matrix, which involves perceiving and reasoning about the properties of multiple items. 4. **Effects of different abstraction levels**: - The researchers proposed three different abstraction levels: entity - level, layout - level, and structural decomposition. Through these abstraction levels, they hope to understand the impact of different levels of abstraction on the analogical reasoning ability of PLMs. - Experimental results show that higher - level abstraction can significantly enhance the analogical reasoning ability of PLMs, especially in complex tasks. ### Main contributions - **Zero - shot analogical reasoning**: The research shows that large - scale PLMs exhibit astonishing analogical reasoning ability under zero - shot conditions, even exceeding human performance and approaching supervised - learning visual methods. - **The role of language abstraction**: The research reveals the importance of language abstraction in supporting analogical reasoning, especially the effectiveness of naming and decomposition strategies. - **The influence of model complexity and prior knowledge**: A detailed analysis shows that model complexity, context learning, and prior knowledge play important roles in solving RPM tasks. ### Conclusion This research provides new ideas for future cognitive architecture design, demonstrates the potential of language in supporting analogical reasoning, and lays the foundation for further exploration of the analogical reasoning ability of AI systems.

In-Context Analogical Reasoning with Pre-Trained Language Models

ARN: Analogical Reasoning on Narratives

AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogies

Relevant or Random: Can LLMs Truly Perform Analogical Reasoning?

From Heuristic to Analytic: Cognitively Motivated Strategies for Coherent Physical Commonsense Reasoning

Semantic Structure-Mapping in LLM and Human Analogical Reasoning

ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base

Evaluating the Robustness of Analogical Reasoning in Large Language Models

Large Language Models as Analogical Reasoners

Beneath Surface Similarity: Large Language Models Make Reasonable Scientific Analogies after Structure Abduction

Using Counterfactual Tasks to Evaluate the Generality of Analogical Reasoning in Large Language Models

Understanding Narratives through Dimensions of Analogy

Why Do We Need Neuro-symbolic AI to Model Pragmatic Analogies?

Inductive Linguistic Reasoning with Large Language Models

How Far Are We from Intelligent Visual Deductive Reasoning?

Learning to reason over visual objects

Analogical Reasoning for Visually Grounded Language Acquisition

Visual-Imagery-Based Analogical Construction in Geometric Matrix Reasoning Task

The Curious Case of Nonverbal Abstract Reasoning with Multi-Modal Large Language Models

Evidence from counterfactual tasks supports emergent analogical reasoning in large language models

Conceptual and Unbiased Reasoning in Language Models