Abstract:A defining characteristic of intelligent systems, whether natural or artificial, is the ability to generalize and infer behaviorally relevant latent causes from high-dimensional sensory input, despite significant variations in the environment. To understand how brains achieve generalization, it is crucial to identify the features to which neurons respond selectively and invariantly. However, the high-dimensional nature of visual inputs, the non-linearity of information processing in the brain, and limited experimental time make it challenging to systematically characterize neuronal tuning and invariances, especially for natural stimuli. Here, we extended "inception loops" - a paradigm that iterates between large-scale recordings, neural predictive models, and in silico experiments followed by in vivo verification - to systematically characterize single neuron invariances in the mouse primary visual cortex. Using the predictive model we synthesized Diverse Exciting Inputs (DEIs), a set of inputs that differ substantially from each other while each driving a target neuron strongly, and verified these DEIs' efficacy in vivo. We discovered a novel bipartite invariance: one portion of the receptive field encoded phase-invariant texture-like patterns, while the other portion encoded a fixed spatial pattern. Our analysis revealed that the division between the fixed and invariant portions of the receptive fields aligns with object boundaries defined by spatial frequency differences present in highly activating natural images. These findings suggest that bipartite invariance might play a role in segmentation by detecting texture-defined object boundaries, independent of the phase of the texture. We also replicated these bipartite DEIs in the functional connectomics MICrONs data set, which opens the way towards a circuit-level mechanistic understanding of this novel type of invariance. Our study demonstrates the power of using a data-driven deep learning approach to systematically characterize neuronal invariances. By applying this method across the visual hierarchy, cell types, and sensory modalities, we can decipher how latent variables are robustly extracted from natural scenes, leading to a deeper understanding of generalization.

An Information Processing Pattern from Robotics Predicts Unknown Properties of the Human Visual System

Inferring Human Vision in a Human-Like Way: Key Factors Influencing the Cognitive Processing of Level-1 Visual Perspective-Taking

A System View of the Recognition and Interpretation of Observed Human Shape, Pose and Action

Passive Attention in Artificial Neural Networks Predicts Human Visual Selectivity

Modeling Visual Information Processing in Brain: A Computer Vision Point of View and Approach

A large-scale examination of inductive biases shaping high-level visual representation in brains and machines

An inverse modeling method to estimate undertain spatial configurations from 2d information and time-based visual discriminations

A Computational Imaging Model and Experimental Assessment for Artificial Visual Prosthesis

Approaching human 3D shape perception with neurally mappable models

Computational Techniques Enabling the Perception of Virtual Images Exclusive to the Retinal Afterimage

Learning to segment self-generated from externally caused optic flow through sensorimotor mismatch circuits

Bionic model of information processing in the retina

The Art of Deception: Color Visual Illusions and Diffusion Models

Modeling the Human Visual System: Comparative Insights from Response-Optimized and Task-Optimized Vision Models, Language Models, and different Readout Mechanisms

Investigating Color Illusions from the Perspective of Computational Color Constancy

Neural Dynamics Model of Visual Decision-Making: Learning from Human Experts

Unidirectional brain-computer interface: Artificial neural network encoding natural images to fMRI response in the visual cortex

A Signal-Processing Neural Model Based on Biological Retina

Visual Illusions Also Deceive Convolutional Neural Networks: Analysis and Implications

Bipartite invariance in mouse primary visual cortex

Generative visual common sense: Testing analysis-by-synthesis on Mondrian-style image.