Abstract:Humans and animals routinely infer relations between different items or events and generalize these relations to novel combinations of items. This allows them to respond appropriately to radically novel circumstances and is fundamental to advanced cognition. However, how learning systems (including the brain) can implement the necessary inductive biases has been unclear. Here we investigated transitive inference (TI), a classic relational task paradigm in which subjects must learn a relation ( > and > ) and generalize it to new combinations of items ( > ). Through mathematical analysis, we found that a broad range of biologically relevant learning models (e.g. gradient flow or ridge regression) perform TI successfully and recapitulate signature behavioral patterns long observed in living subjects. First, we found that models with item-wise additive representations automatically encode transitive relations. Second, for more general representations, a single scalar “conjunctivity factor” determines model behavior on TI and, further, the principle of norm minimization (a standard statistical inductive bias) enables models with fixed, partly conjunctive representations to generalize transitively. Finally, neural networks in the “rich regime,” which enables representation learning and has been found to improve generalization, unexpectedly show poor generalization and anomalous behavior. We find that such networks implement a form of norm minimization (over hidden weights) that yields a local encoding mechanism lacking transitivity. Our findings show how minimal statistical learning principles give rise to a classical relational inductive bias (transitivity), explain empirically observed behaviors, and establish a formal approach to understanding the neural basis of relational abstraction.

Learning abstract visual concepts via probabilistic program induction in a Language of Thought

Human-level concept learning through probabilistic program induction

Human-like Few-Shot Learning via Bayesian Reasoning over Natural Language

People infer recursive visual concepts from just a few examples

Learning to Infer Generative Template Programs for Visual Concepts

Visual Concept Learning: Combining Machine Vision and Bayesian Generalization on Concept Hierarchies

Inference of Abstraction for a Unified Account of Reasoning and Learning

The learnability of abstract syntactic principles

A mathematical theory of relational generalization in transitive inference

Learning abstract structure for drawing by efficient motor program induction

On the hazards of relating representations and inductive biases

From Concrete to Abstract: A Multimodal Generative Approach to Abstract Concept Learning

Variable Assignment Invariant Neural Networks for Learning Logic Programs

Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement

Probabilistic programming versus meta-learning as models of cognition

Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning

A Concept Learning Approach to Multisensory Object Perception

How a Minimal Learning Agent can Infer the Existence of Unobserved Variables in a Complex Environment

Learning Discrete Concepts in Latent Hierarchical Models

Towards Understanding How Machines Can Learn Causal Overhypotheses

Abstract representations of events arise from mental errors in learning and memory