Abstract:Neurons throughout the brain modulate their firing rate lawfully in response to sensory input. Theories of neural computation posit that these modulations reflect the outcome of a constrained optimization in which neurons aim to robustly and efficiently represent sensory information. Our understanding of how this optimization varies across different areas in the brain, however, is still in its infancy. Here, we show that neural sensory responses transform along the dorsal stream of the visual system in a manner consistent with a transition from optimizing for information preservation towards optimizing for perceptual discrimination. Focusing on the representation of binocular disparities—the slight differences in the retinal images of the two eyes—we re-analyze measurements characterizing neuronal tuning curves in brain areas V1, V2, and MT (middle temporal) in the macaque monkey. We compare these to measurements of the statistics of binocular disparity typically encountered during natural behaviors using a Fisher Information framework. The differences in tuning curve characteristics across areas are consistent with a shift in optimization goals: V1 and V2 population-level responses are more consistent with maximizing the information encoded about naturally occurring binocular disparities, while MT responses shift towards maximizing the ability to support disparity discrimination. We find that a change towards tuning curves preferring larger disparities is a key driver of this shift. These results provide new insight into previously-identified differences between disparity-selective areas of cortex and suggest these differences play an important role in supporting visually-guided behavior. Our findings emphasize the need to consider not just information preservation and neural resources, but also relevance to behavior, when assessing the optimality of neural codes. The nervous system needs to transform information from the sensory organs into signals that can be used to guide behavior. Neural activity is noisy and can consume large amount of energy, so sensory neurons must optimize their information processing so as to limit energy consumption while maintaining key behaviorally-relevant information. In this report, we re-examine classically-defined brain areas in the visual processing hierarchy, and ask whether neurons in these areas vary lawfully in how they represent sensory information. Our results suggest that neurons in these brain areas shift from being an optimal conduit for conveying sensory information towards prioritizing information that supports key perceptual discriminations during natural tasks.

Neural and perceptual signatures of efficient sensory coding

Transformations of sensory information in the brain suggest changing criteria for optimality

Applying the Efficient Coding Principle to Understand Encoding of Multisensory and Multimodality Sensory Signals

Efficiency turns the table on neural encoding, decoding and noise

MSE-based analysis of optimal tuning functions predicts phenomena observed in sensory neurons

Efficient coding explains neural response homeostasis and stimulus-specific adaptation

Sensory perception relies on fitness-maximizing codes

Efficient Coding As the Provenance of Matched and Opposite Neuronal Feature Preferences for Multisensory and Multi-Modal Inputs

Efficient representation as a design principle for neural coding and computation

A Unified Theory of Response Sparsity and Variability for Energy-Efficient Neural Coding

Jointly efficient encoding and decoding in neural populations

Bayesian Efficient Coding

Parsimony, exhaustivity and balanced detection in neocortex

Weighting neurons by selectivity produces near-optimal population codes

Energy‐efficient Neural Information Processing in Individual Neurons and Neuronal Networks

Optimal Encoding in Stochastic Latent-Variable Models

Closed-loop estimation of retinal network sensitivity reveals signature of efficient coding

Efficient population coding of sensory stimuli

The geometry of correlated variability leads to highly suboptimal discriminative sensory coding

Neuronal Selectivity, Population Sparseness, and Ergodicity in the Inferior Temporal Visual Cortex.

Revealing nonlinear neural decoding by analyzing choices