Abstract:Humans leverage compositionality to efficiently learn new concepts, understanding how familiar parts can combine together to form novel objects. In contrast, popular computer vision models struggle to make the same types of inferences, requiring more data and generalizing less flexibly than people do. Here, we study these distinctively human abilities across a range of different types of visual composition, examining how people classify and generate ``alien figures'' with rich relational structure. We also develop a Bayesian program induction model which searches for the best programs for generating the candidate visual figures, utilizing a large program space containing different compositional mechanisms and abstractions. In few shot classification tasks, we find that people and the program induction model can make a range of meaningful compositional generalizations, with the model providing a strong account of the experimental data as well as interpretable parameters that reveal human assumptions about the factors invariant to category membership (here, to rotation and changing part attachment). In few shot generation tasks, both people and the models are able to construct compelling novel examples, with people behaving in additional structured ways beyond the model capabilities, e.g. making choices that complete a set or reconfiguring existing parts in highly novel ways. To capture these additional behavioral patterns, we develop an alternative model based on neuro-symbolic program induction: this model also composes new concepts from existing parts yet, distinctively, it utilizes neural network modules to successfully capture residual statistical structure. Together, our behavioral and computational findings show how people and models can produce a rich variety of compositional behavior when classifying and generating visual objects.

Generating new concepts with hybrid neuro-symbolic models

Learning Evolved Combinatorial Symbols with a Neuro-symbolic Generative Model

Compositional diversity in visual concept learning

Distilling Symbolic Priors for Concept Learning into Neural Networks

A Brain-inspired Computational Model for Human-like Concept Learning

A Study of Compositional Generalization in Neural Models

A Neural Network Model for Concept Formation

Achieving Data Efficient Neural Networks with Hybrid Concept-based Models

Human-level concept learning through probabilistic program induction

Learning to Infer Generative Template Programs for Visual Concepts

Training neural networks to encode symbols enables combinatorial generalization

Optimisation in Neurosymbolic Learning Systems

Formal Conceptual Views in Neural Networks

NeSyCoCo: A Neuro-Symbolic Concept Composer for Compositional Generalization

Explainable Concept Generation through Vision-Language Preference Learning

Alien Recombination: Exploring Concept Blends Beyond Human Cognitive Availability in Visual Art

Neurosymbolic Models for Computer Graphics

Neuro-Symbolic Generative Art: A Preliminary Study

A Neuro-mimetic Realization of the Common Model of Cognition via Hebbian Learning and Free Energy Minimization

Seeking the Building Blocks of Visual Imagery and Creativity in a Cognitively Inspired Neural Network