Abstract:Humans and other primates can rapidly detect familiar objects and distinguish them from never-before-seen novel objects. We have an astonishing capacity to remember the details of visual scenes even after a single, fleeting experience. This ability is thought to rely in part on experience-dependent changes in the primate inferotemporal cortex (IT). Single neurons in IT encode visual familiarity by discriminating between novel and familiar stimuli, with stronger neural activity on average for novel images. However, key open questions are to understand how neural encoding in IT changes as images progress from novel to highly familiar, and what learning rules and computations can account for learning-dependent changes in IT activity. Here, we investigate the timescales over which novel stimuli become familiar by recording in IT as initially novel images become increasingly familiar both within and across days. We identified salient and persistent memory-related signals in IT that spanned multiple timescales of minutes, hours, and days. Average neural activity progressively decreased with familiarity as firing rates were strongest for novel, weaker for intermediately familiar, and weakest for highly familiar images. Neural signatures of familiarity learning were slow to develop as response reductions to initially-novel images emerged gradually over multiple days (or hundreds of views) of visual experience. In addition to slow changes that emerged across sessions, neural responses to novel images showed rapid decreases with familiarity within single sessions. To gain insight into the mechanisms underlying changes of visual responses with familiarity, we use computational modeling to investigate which plasticity rules are consistent with these changes. Learning rules inferred from the neural data revealed a strong diversity with many neurons following a ‘negative’ plasticity rule as they exhibited synaptic depression over the course of learning across multiple days. A recurrent network model with two plasticity time constants – a slow time constant for long timescales and a fast time constant for short timescales – captured key dynamic features accompanying the transition from novel to familiar, including a gradual decrease in firing rates over multiple sessions, and a rapid decrease in firing rates within single sessions. Our findings suggest that distinct and complementary plasticity rules operating at different timescales may underlie the inferotemporal code for visual familiarity.

Manifold Transform by Recurrent Cortical Circuit Enhances Robust Encoding of Familiar Stimuli

Functional organization and natural scene responses across mouse visual cortical areas revealed with encoding manifolds

Network Model Of Top-Down Influences On Local Gain And Contextual Interactions In Visual Cortex

Visual familiarity learning at multiple timescales in the primate inferotemporal cortex

Recurrent Feedback Improves Feedforward Representations in Deep Neural Networks

Fundamental principles of cortical computation: unsupervised learning with prediction, compression and feedback

A recurrent neural circuit in deblurs visual inputs

Recurrent networks improve neural response prediction and provide insights into underlying cortical circuits

Population encoding of stimulus features along the visual hierarchy

Exploring the Architectural Biases of the Canonical Cortical Microcircuit

Functional Connectivity among Cortical Regions is Shaped by Associative Experiences

Pattern completion and disruption characterize contextual modulation in the visual cortex

The logic of recurrent circuits in the primary visual cortex

Evidence that recurrent circuits are critical to the ventral stream’s execution of core object recognition behavior

Emergence of V1 connectivity pattern and Hebbian rule in a performance-optimized artificial neural network

Top-Down Modulation of Lateral Interactions in Visual Cortex

Contrast-dependent response modulation in convolutional neural networks captures behavioral and neural signatures of visual adaptation

Neural manifolds in V1 change with top-down signals from V4 targeting the foveal region

Task-Driven Convolutional Recurrent Models of the Visual System

Cortical Motion Perception Emerges from Dimensionality Reduction with Evolved Spike-Timing-Dependent Plasticity Rules

Linking Neural Manifolds to Circuit Structure in Recurrent Networks