Abstract:Neurons throughout the brain modulate their firing rate lawfully in response to sensory input. Theories of neural computation posit that these modulations reflect the outcome of a constrained optimization in which neurons aim to robustly and efficiently represent sensory information. Our understanding of how this optimization varies across different areas in the brain, however, is still in its infancy. Here, we show that neural sensory responses transform along the dorsal stream of the visual system in a manner consistent with a transition from optimizing for information preservation towards optimizing for perceptual discrimination. Focusing on the representation of binocular disparities—the slight differences in the retinal images of the two eyes—we re-analyze measurements characterizing neuronal tuning curves in brain areas V1, V2, and MT (middle temporal) in the macaque monkey. We compare these to measurements of the statistics of binocular disparity typically encountered during natural behaviors using a Fisher Information framework. The differences in tuning curve characteristics across areas are consistent with a shift in optimization goals: V1 and V2 population-level responses are more consistent with maximizing the information encoded about naturally occurring binocular disparities, while MT responses shift towards maximizing the ability to support disparity discrimination. We find that a change towards tuning curves preferring larger disparities is a key driver of this shift. These results provide new insight into previously-identified differences between disparity-selective areas of cortex and suggest these differences play an important role in supporting visually-guided behavior. Our findings emphasize the need to consider not just information preservation and neural resources, but also relevance to behavior, when assessing the optimality of neural codes. The nervous system needs to transform information from the sensory organs into signals that can be used to guide behavior. Neural activity is noisy and can consume large amount of energy, so sensory neurons must optimize their information processing so as to limit energy consumption while maintaining key behaviorally-relevant information. In this report, we re-examine classically-defined brain areas in the visual processing hierarchy, and ask whether neurons in these areas vary lawfully in how they represent sensory information. Our results suggest that neurons in these brain areas shift from being an optimal conduit for conveying sensory information towards prioritizing information that supports key perceptual discriminations during natural tasks.

Efficient representation as a design principle for neural coding and computation

Neural and perceptual signatures of efficient sensory coding

Bayesian Efficient Coding

Efficiency turns the table on neural encoding, decoding and noise

Sensory perception relies on fitness-maximizing codes

Transformations of sensory information in the brain suggest changing criteria for optimality

Applying the Efficient Coding Principle to Understand Encoding of Multisensory and Multimodality Sensory Signals

Energy‐efficient Neural Information Processing in Individual Neurons and Neuronal Networks

Neural and phenotypic representation under the free-energy principle

Neuromorphic sensory systems

Efficient Coding As the Provenance of Matched and Opposite Neuronal Feature Preferences for Multisensory and Multi-Modal Inputs

Efficient coding explains neural response homeostasis and stimulus-specific adaptation

Neural coding in the visual system of Drosophila melanogaster: How do small neural populations support visually guided behaviours?

Jointly efficient encoding and decoding in neural populations

Efficient human-like semantic representations via the Information Bottleneck principle

Parsimony, exhaustivity and balanced detection in neocortex

Neural Elements for Predictive Coding

An evolutionary perspective on the design of neuromorphic shape filters

Intelligent encoding and economical communication in the visual stream

Privileged representational axes in biological and artificial neural networks

A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNs