Abstract:The visual system is hierarchically organized to process visual information in successive stages. Neural representations vary drastically across the first stages of visual processing: at the output of the retina, ganglion cell receptive fields (RFs) exhibit a clear antagonistic center-surround structure, whereas in the primary visual cortex, typical RFs are sharply tuned to a precise orientation. There is currently no unified theory explaining these differences in representations across layers. Here, using a deep convolutional neural network trained on image recognition as a model of the visual system, we show that such differences in representation can emerge as a direct consequence of different neural resource constraints on the retinal and cortical networks, and we find a single model from which both geometries spontaneously emerge at the appropriate stages of visual processing. The key constraint is a reduced number of neurons at the retinal output, consistent with the anatomy of the optic nerve as a stringent bottleneck. Second, we find that, for simple cortical networks, visual representations at the retinal output emerge as nonlinear and lossy feature detectors, whereas they emerge as linear and faithful encoders of the visual scene for more complex cortices. This result predicts that the retinas of small vertebrates should perform sophisticated nonlinear computations, extracting features directly relevant to behavior, whereas retinas of large animals such as primates should mostly encode the visual scene linearly and respond to a much broader range of stimuli. These predictions could reconcile the two seemingly incompatible views of the retina as either performing feature extraction or efficient coding of natural scenes, by suggesting that all vertebrates lie on a spectrum between these two objectives, depending on the degree of neural resources allocated to their visual system.

Revealing Fine Structures of the Retinal Receptive Field by Deep-Learning Networks

Revealing structure components of the retina by deep learning networks

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

CSNN: an Augmented Spiking Based Framework with Perceptron-Inception

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Convolutional Neural Networks Exploiting Attributes of Biological Neurons

Unraveling Neural Coding of Dynamic Natural Visual Scenes Via Convolutional Recurrent Neural Networks.

A Retina-LGN-V1 Structure-like Spiking Neuron Network for Image Feature Extraction

Deeper Interpretability of Deep Networks

Modeling responses of macaque and human retinal ganglion cells to natural images using a convolutional neural network

Incremental Learning and Self-Attention Mechanisms Improve Neural System Identification

Structured Receptive Fields in CNNs

Convolutional neural networks for vision neuroscience: significance, developments, and outstanding issues

From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction

Neural Encoding and Decoding with Deep Learning for Dynamic Natural Vision

A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNs

A Comparison of Statistical Learning of Naturalistic Textures Between DCNNs and the Human Visual Hierarchy

Deep Pulse-Coupled Neural Networks

Explaining V1 Properties with a Biologically Constrained Deep Learning Architecture

Study Of The Application Of Deep Convolutional Neural Networks (Cnns) In Processing Sensor Data And Biomedical Images

Neural encoding with unsupervised spiking convolutional neural network