Abstract:Neurons in the primary visual cortex respond selectively to simple features of visual stimuli, such as orientation and spatial frequency. Simple cells, which have phase-sensitive responses, can be modeled by a single receptive field filter in a linear-nonlinear model. However, it is challenging to analyze phase-invariant complex cells, which require more elaborate models having a combination of nonlinear subunits. Estimating parameters of these models is made additionally more difficult by cortical neurons' trial-to-trial response variability. We develop a simple convolutional neural network method to estimate receptive field models for both simple and complex visual cortex cells from their responses to natural images. The model consists of a spatiotemporal filter, a parameterized rectifier unit (PReLU), and a two-dimensional Gaussian "map" of the receptive field envelope. A single model parameter determines the simple vs. complex nature of the receptive field, capturing complex cell responses as a summation of homogeneous subunits, and collapsing to a linear-nonlinear model for simple type cells. The convolutional method predicts simple and complex cell responses to natural image stimuli as well as grating tuning curves. The fitted models yield a continuum of values for the PReLU parameter across the sampled neurons, showing that the simple/complex nature of cells can vary in a continuous manner. We demonstrate that complex-like cells respond less reliably than simple-like cells. However, compensation for this unreliability with noise ceiling analysis reveals predictive performance for complex cells proportionately closer to that for simple cells. Most spatial receptive field structures are well fit by Gabor functions, whose parameters confirm well-known properties of cat A17/18 receptive fields. Methods for recording increasingly many visual cortex neurons are advancing rapidly, demanding new approaches to characterize diverse receptive fields. We present a compact convolutional neural network model of early cortical neurons, which is uniformly applicable to simple and complex cells, and whose parameters are straightforwardly interpretable and readily estimated from responses to natural image stimuli. This novel approach introduces a single estimated model parameter to capture the simple/complex nature of a neuron's receptive field, revealing a continuum of simple vs complex-like behaviour. We show that almost all complex-like cells exhibit a lower response reliability to repeated presentations of the same stimuli, compared to more reliable responses from simple-like cells. Accounting for this "noise ceiling" brings predictive performance for complex cells proportionately closer to that for simple cells. Using this model estimation approach with natural images, we evaluate findings from previous approaches that were restricted to simple-type cells and the use of grating or white noise stimuli, revealing a diversity of Gabor-like spatial receptive field shapes, which lie along a continuum of spatial bandwidths.

Convolutional neural network models of V1 responses to complex patterns

Convolutional neural network models of neuronal responses in macaque V1 reveal limited non-linear processing

Convolutional neural network models applied to neuronal responses in macaque V1 reveal limited nonlinear processing

Diverse task-driven modeling of macaque V4 reveals functional specialization towards semantic tasks

Modeling responses of macaque and human retinal ganglion cells to natural images using a convolutional neural network

Two-photon Imaging of V1 Responses to Complex Stimulus Patterns in Awake Macaque Monkeys

Complexity and diversity in sparse code priors improve receptive field characterization of Macaque V1 neurons

Convolutional neural network models describe the encoding subspace of local circuits in auditory cortex

Complex Pattern Selectivity in Macaque Primary Visual Cortex Revealed by Large-Scale Two-Photon Imaging.

Explaining V1 Properties with a Biologically Constrained Deep Learning Architecture

Foundations of visual form selectivity for neurons in macaque V1 and V2

A Rich Source of Labels for Deep Network Models of the Primate Dorsal Visual Stream

Revealing Fine Structures of the Retinal Receptive Field by Deep-Learning Networks

Decoding Pixel-Level Image Features From Two-Photon Calcium Signals of Macaque Visual Cortex

Multimodal Deep Learning Model Unveils Behavioral Dynamics of V1 Activity in Freely Moving Mice

Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future

Large-scale two-photon imaging revealed super-sparse population codes in V1 superficial layer of awake monkeys

Explaining face representation in the primate brain using different computational models

Convolutional neural networks for vision neuroscience: significance, developments, and outstanding issues

Model-based characterization of the selectivity of neurons in primary visual cortex

Estimating receptive fields of simple and complex cells in early visual cortex: A convolutional neural network model with parameterized rectification