Abstract:A fundamental component of human vision is our ability to parse complex visual scenes and judge the relations between their constituent objects. AI benchmarks for visual reasoning have driven rapid progress in recent years with state-of-the-art systems now reaching human accuracy on some of these benchmarks. Yet, a major gap remains in terms of the sample efficiency with which humans and AI systems learn new visual reasoning tasks. Humans' remarkable efficiency at learning has been at least partially attributed to their ability to harness compositionality -- such that they can efficiently take advantage of previously gained knowledge when learning new tasks. Here, we introduce a novel visual reasoning benchmark, Compositional Visual Relations (CVR), to drive progress towards the development of more data-efficient learning algorithms. We take inspiration from fluidic intelligence and non-verbal reasoning tests and describe a novel method for creating compositions of abstract rules and associated image datasets at scale. Our proposed benchmark includes measures of sample efficiency, generalization and transfer across task rules, as well as the ability to leverage compositionality. We systematically evaluate modern neural architectures and find that, surprisingly, convolutional architectures surpass transformer-based architectures across all performance measures in most data regimes. However, all computational models are a lot less data efficient compared to humans even after learning informative visual representations using self-supervision. Overall, we hope that our challenge will spur interest in the development of neural architectures that can learn to harness compositionality toward more efficient learning.

Conditional Kronecker Batch Normalization for Compositional Reasoning.

Compositional Substitutivity of Visual Reasoning for Visual Question Answering

Compositional Kronecker Context Optimization for Vision-Language Models

BCN: Batch Channel Normalization for Image Classification

Linguistically Driven Graph Capsule Network for Visual Question Reasoning

Cross-Iteration Batch Normalization

Multi-Level Knowledge Injecting for Visual Commonsense Reasoning

Triple-CFN: Restructuring Concept Spaces for Enhancing Abstract Reasoning Process

Unified Batch Normalization: Identifying and Alleviating the Feature Condensation in Batch Normalization and a Unified Framework

From Shallow to Deep: Compositional Reasoning over Graphs for Visual Question Answering

Batch Normalization and the impact of batch structure on the behavior of deep convolution networks

Connective Cognition Network for Directional Visual Commonsense Reasoning.

Decorrelated Batch Normalization

Joint Answering and Explanation for Visual Commonsense Reasoning

COVR: A test-bed for Visually Grounded Compositional Generalization with real images

A Benchmark for Compositional Visual Reasoning

Compositional Generalization by Learning Analytical Expressions.

Restructuring Batch Normalization to Accelerate CNN Training

A Study of Compositional Generalization in Neural Models

Curriculum Learning for Compositional Visual Reasoning

Neural Compositional Rule Learning for Knowledge Graph Reasoning