Abstract:In everyday life, we integrate visual and auditory information in routine tasks such as navigation and communication. While concurrent sound can improve visual perception, the neuronal correlates of audiovisual integration are not fully understood. Specifically, it remains unclear whether neuronal firing patters in the primary visual cortex (V1) of awake animals demonstrate similar sound-induced improvement in visual discriminability. Furthermore, presentation of sound is associated with movement in the subjects, but little is understood about whether and how sound-associated movement affects audiovisual integration in V1. Here, we investigated how sound and movement interact to modulate V1 visual responses in awake, head-fixed mice and whether this interaction improves neuronal encoding of the visual stimulus. We presented visual drifting gratings with and without simultaneous auditory white noise to awake mice while recording mouse movement and V1 neuronal activity. Sound modulated activity of 80% of light-responsive neurons, with 95% of neurons increasing activity when the auditory stimulus was present. A generalized linear model (GLM) revealed that sound and movement had distinct and complementary effects of the neuronal visual responses. Furthermore, decoding of the visual stimulus from the neuronal activity was improved with sound, an effect that persisted even when controlling for movement. These results demonstrate that sound and movement modulate visual responses in complementary ways, improving neuronal representation of the visual stimulus. This study clarifies the role of movement as a potential confound in neuronal audiovisual responses and expands our knowledge of how multimodal processing is mediated at a neuronal level in the awake brain.SIGNIFICANCE STATEMENT Sound and movement are both known to modulate visual responses in the primary visual cortex; however, sound-induced movement has largely remained unaccounted for as a potential confound in audiovisual studies in awake animals. Here, authors found that sound and movement both modulate visual responses in an important visual brain area, the primary visual cortex, in distinct, yet complementary ways. Furthermore, sound improved encoding of the visual stimulus even when accounting for movement. This study reconciles contrasting theories on the mechanism underlying audiovisual integration and asserts the primary visual cortex as a key brain region participating in tripartite sensory interactions.

Enhancement of Visual Motion Detection Thresholds in Early Deaf People

The Right Hemisphere Planum Temporale Supports Enhanced Visual Motion Detection Ability in Deaf People: Evidence from Cortical Thickness

Reorganization of the auditory, visual and multimodal areas in early deaf individuals

Long-term Deficits in Motion Detection Thresholds and Spike Count Variability after Unilateral Vestibular Lesion

A Wearable Vision-To-Audio Sensory Substitution Device for Blind Assistance and the Correlated Neural Substrates

Auditory areas are recruited for naturalistic visual meaning in early deaf people

Responses in area hMT+ reflect tuning for both auditory frequency and motion after blindness early in life

Audio-visual interaction in visual motion detection: Synchrony versus Asynchrony

Enhanced spontaneous functional connectivity of the superior temporal gyrus in early deafness

Auditory perception and the control of spatially coordinated action of deaf and hearing children

The neural organization of visual information in the auditory cortex of the congenitally deaf

Auditory motion affects visual biological motion processing

Does Congenital Deafness Affect the Structural and Functional Architecture of Primary Visual Cortex?

Neural adaptations to congenital deafness: enhanced tactile discrimination through cross-modal neural plasticity - an fMRI study

Differences in biological motion perception associated with hearing status and age of signed language exposure.

Increments in Visual Motion Coherence Are More Readily Detected Than Decrements

An investigation of multisensory perception of surrounding space in aided congenitally hearing impaired

Motion Perceptual Learning in Noise Improves Neural Sensitivity in Human MT+ and IPS

Cross-modal integration of auditory and visual motion signals

Sound Improves Neuronal Encoding of Visual Stimuli in Mouse Primary Visual Cortex

Task-specific reorganization of the auditory cortex in deaf humans