Abstract:Neurons throughout the brain modulate their firing rate lawfully in response to sensory input. Theories of neural computation posit that these modulations reflect the outcome of a constrained optimization in which neurons aim to robustly and efficiently represent sensory information. Our understanding of how this optimization varies across different areas in the brain, however, is still in its infancy. Here, we show that neural sensory responses transform along the dorsal stream of the visual system in a manner consistent with a transition from optimizing for information preservation towards optimizing for perceptual discrimination. Focusing on the representation of binocular disparities—the slight differences in the retinal images of the two eyes—we re-analyze measurements characterizing neuronal tuning curves in brain areas V1, V2, and MT (middle temporal) in the macaque monkey. We compare these to measurements of the statistics of binocular disparity typically encountered during natural behaviors using a Fisher Information framework. The differences in tuning curve characteristics across areas are consistent with a shift in optimization goals: V1 and V2 population-level responses are more consistent with maximizing the information encoded about naturally occurring binocular disparities, while MT responses shift towards maximizing the ability to support disparity discrimination. We find that a change towards tuning curves preferring larger disparities is a key driver of this shift. These results provide new insight into previously-identified differences between disparity-selective areas of cortex and suggest these differences play an important role in supporting visually-guided behavior. Our findings emphasize the need to consider not just information preservation and neural resources, but also relevance to behavior, when assessing the optimality of neural codes. The nervous system needs to transform information from the sensory organs into signals that can be used to guide behavior. Neural activity is noisy and can consume large amount of energy, so sensory neurons must optimize their information processing so as to limit energy consumption while maintaining key behaviorally-relevant information. In this report, we re-examine classically-defined brain areas in the visual processing hierarchy, and ask whether neurons in these areas vary lawfully in how they represent sensory information. Our results suggest that neurons in these brain areas shift from being an optimal conduit for conveying sensory information towards prioritizing information that supports key perceptual discriminations during natural tasks.

The neural basis for violations of Weber’s law in self-motion perception

Transformation of Spatiotemporal Dynamics in the Macaque Vestibular System from Otolith Afferents to Cortex

Neuronal Thresholds and Choice-Related Activity of Otolith Afferent Fibers During Heading Perception

Long-term Deficits in Motion Detection Thresholds and Spike Count Variability after Unilateral Vestibular Lesion

Neural populations within macaque early vestibular pathways are adapted to encode natural self-motion

Weber's law and natural inference

Violated predictions enhance the representational fidelity of visual features in perception

Decision-making and Weber's Law: a Neurophysiological Model.

The precision of signals encoding active self-movement

Dynamic Reweighting of Visual and Vestibular Cues during Self-Motion Perception

Cortical Mechanisms of Multisensory Linear Self-motion Perception

The Uncertainty Associated with Visual Flow Fields and Their Influence on Postural Sway: Weber's Law Suffices to Explain the Nonlinearity of Vection

The implications of perception as probabilistic inference for correlated neural variability during behavior

Neural correlates of perisaccadic visual mislocalization in extrastriate cortex

Accumulation of Inertial Sensory Information in the Perception of Whole Body Yaw Rotation

Humans gradually integrate sudden gain or loss of visual information into spatial orientation perception

Violation of expectation: neural correlates reflect bases of prediction

Learn to integrate parts for whole through correlated neural variability

Spatial Heterogeneity of Biological Motion Perception

Transformations of sensory information in the brain suggest changing criteria for optimality

The role of self-motion processing in impairments of the spatial perception of auditory scenes