Abstract:In everyday life, we integrate visual and auditory information in routine tasks such as navigation and communication. While concurrent sound can improve visual perception, the neuronal correlates of audiovisual integration are not fully understood. Specifically, it remains unclear whether neuronal firing patters in the primary visual cortex (V1) of awake animals demonstrate similar sound-induced improvement in visual discriminability. Furthermore, presentation of sound is associated with movement in the subjects, but little is understood about whether and how sound-associated movement affects audiovisual integration in V1. Here, we investigated how sound and movement interact to modulate V1 visual responses in awake, head-fixed mice and whether this interaction improves neuronal encoding of the visual stimulus. We presented visual drifting gratings with and without simultaneous auditory white noise to awake mice while recording mouse movement and V1 neuronal activity. Sound modulated activity of 80% of light-responsive neurons, with 95% of neurons increasing activity when the auditory stimulus was present. A generalized linear model (GLM) revealed that sound and movement had distinct and complementary effects of the neuronal visual responses. Furthermore, decoding of the visual stimulus from the neuronal activity was improved with sound, an effect that persisted even when controlling for movement. These results demonstrate that sound and movement modulate visual responses in complementary ways, improving neuronal representation of the visual stimulus. This study clarifies the role of movement as a potential confound in neuronal audiovisual responses and expands our knowledge of how multimodal processing is mediated at a neuronal level in the awake brain.SIGNIFICANCE STATEMENT Sound and movement are both known to modulate visual responses in the primary visual cortex; however, sound-induced movement has largely remained unaccounted for as a potential confound in audiovisual studies in awake animals. Here, authors found that sound and movement both modulate visual responses in an important visual brain area, the primary visual cortex, in distinct, yet complementary ways. Furthermore, sound improved encoding of the visual stimulus even when accounting for movement. This study reconciles contrasting theories on the mechanism underlying audiovisual integration and asserts the primary visual cortex as a key brain region participating in tripartite sensory interactions.

Triple dissociation of visual, auditory and motor processing in mouse primary visual cortex

Stability of cross-sensory input to primary somatosensory cortex across experience

A primary sensory cortical interareal feedforward inhibitory circuit for tacto-visual integration

Modulation of auditory responses by visual inputs in the mouse auditory cortex

Higher Order Visual Areas Enhance Stimulus Responsiveness in Mouse Primary Visual Cortex

Higher-order visual areas broaden stimulus responsiveness in mouse primary visual cortex

Sound Improves Neuronal Encoding of Visual Stimuli in Mouse Primary Visual Cortex

Visual experience-dependent auditory and visual plasticity in the mouse visual cortex

The auditory-visual multisensory neurons and auditory-visual information integration in rat cortex

Combined diffusion-weighted and functional magnetic resonance imaging reveals a temporal-occipital network involved in auditory-visual object processing

Auditory cortex tracks both auditory and visual stimulus dynamics using low-frequency neuronal phase modulation.

Cross-modal Representations in Early Visual and Auditory Cortices Revealed by Multi-Voxel Pattern Analysis.

Convergence of visual and whisker responses in the primary somatosensory thalamus (ventral posterior medial region) of the mouse

Multimodal mismatch responses in mouse auditory cortex

A Double Dissociation between Anterior and Posterior Superior Temporal Gyrus for Processing Audiovisual Speech Demonstrated by Electrocorticography

Parallel Processing of Sound Dynamics Across Mouse Auditory Cortex Via Spatially Patterned Thalamic Inputs and Distinct Areal Intracortical Circuits

Anatomical Evidence of Multimodal Integration in Primate Striate Cortex

Audiovisual stimuli enhance narrowband gamma activity along the mouse thalamocortical visual circuit

Laminar organization of visual responses in core and parabelt auditory cortex

Visually-evoked activity and variable modulation of auditory responses in the macaque inferior colliculus

Mouse Primary Visual Cortex Is Used to Detect Both Orientation and Contrast Changes