Abstract:A defining characteristic of intelligent systems, whether natural or artificial, is the ability to generalize and infer behaviorally relevant latent causes from high-dimensional sensory input, despite significant variations in the environment. To understand how brains achieve generalization, it is crucial to identify the features to which neurons respond selectively and invariantly. However, the high-dimensional nature of visual inputs, the non-linearity of information processing in the brain, and limited experimental time make it challenging to systematically characterize neuronal tuning and invariances, especially for natural stimuli. Here, we extended "inception loops" - a paradigm that iterates between large-scale recordings, neural predictive models, and in silico experiments followed by in vivo verification - to systematically characterize single neuron invariances in the mouse primary visual cortex. Using the predictive model we synthesized Diverse Exciting Inputs (DEIs), a set of inputs that differ substantially from each other while each driving a target neuron strongly, and verified these DEIs' efficacy in vivo. We discovered a novel bipartite invariance: one portion of the receptive field encoded phase-invariant texture-like patterns, while the other portion encoded a fixed spatial pattern. Our analysis revealed that the division between the fixed and invariant portions of the receptive fields aligns with object boundaries defined by spatial frequency differences present in highly activating natural images. These findings suggest that bipartite invariance might play a role in segmentation by detecting texture-defined object boundaries, independent of the phase of the texture. We also replicated these bipartite DEIs in the functional connectomics MICrONs data set, which opens the way towards a circuit-level mechanistic understanding of this novel type of invariance. Our study demonstrates the power of using a data-driven deep learning approach to systematically characterize neuronal invariances. By applying this method across the visual hierarchy, cell types, and sensory modalities, we can decipher how latent variables are robustly extracted from natural scenes, leading to a deeper understanding of generalization.

Recurrent pattern completion drives the neocortical representation of sensory inference

Pattern completion and disruption characterize contextual modulation in the visual cortex

Stimulus history, not expectation, drives sensory prediction errors in mammalian cortex

Recurrent cortical networks encode natural sensory statistics via sequence filtering

The logic of recurrent circuits in the primary visual cortex

Cooperative thalamocortical circuit mechanism for sensory prediction errors

Recurrent infomax generates cell assemblies, avalanches, and simple cell-like selectivity

Neocortical inhibitory imbalance predicts successful sensory detection

Cortical reactivations predict future sensory responses

A role for recurrent processing in object completion: neurophysiological, psychophysical and computational"evidence

A latent pool of neurons silenced by sensory-evoked inhibition can be recruited to enhance perception

A recurrent neural circuit in deblurs visual inputs

Experience-dependent predictions of feedforward and contextual information in mouse visual cortex

Parsimony, exhaustivity and balanced detection in neocortex

Cortical networks with multiple interneuron types generate oscillatory patterns during predictive coding

Recurrent inhibition refines mental templates to optimize perceptual decisions

Bipartite invariance in mouse primary visual cortex

Learned response dynamics reflect stimulus timing and encode temporal expectation violations in superficial layers of mouse V1

Predictive learning rules generate a cortical-like replay of probabilistic sensory experiences

Correlated spontaneous activity sets up multi-sensory integration in the developing higher-order cortex

Coherent cortical representations develop after experience via feedforward-recurrent circuit alignment