Abstract:Our brain perceives the world by exploiting multiple sensory modalities to extract information about various aspects of external stimuli. If these sensory cues are from the same stimulus of interest, they should be integrated to improve perception; otherwise, they should be segregated to distinguish different stimuli. In reality, however, the brain faces the challenge of recognizing stimuli without knowing in advance whether sensory cues come from the same or different stimuli. To address this challenge and to recognize stimuli rapidly, we argue that the brain should carry out multisensory integration and segregation concurrently with complementary neuron groups. Studying an example of inferring heading-direction via visual and vestibular cues, we develop a concurrent multisensory processing neural model which consists of two reciprocally connected modules, the dorsal medial superior temporal area (MSTd) and the ventral intraparietal area (VIP), and that at each module, there exists two distinguishing groups of neurons, congruent and opposite neurons. Specifically, congruent neurons implement cue integration, while opposite neurons compute the cue disparity, both optimally as described by Bayesian inference. The two groups of neurons provide complementary information which enables the neural system to assess the validity of cue integration and, if necessary, to recover the lost information associated with individual cues without re-gathering new inputs. Through this process, the brain achieves rapid stimulus perception if the cues come from the same stimulus of interest, and differentiates and recognizes stimuli based on individual cues with little time delay if the cues come from different stimuli of interest. Our study unveils the indispensable role of opposite neurons in multisensory processing and sheds light on our understanding of how the brain achieves multisensory processing efficiently and rapidly. Significance Statement Our brain perceives the world by exploiting multiple sensory cues. These cues need to be integrated to improve perception if they come from the same stimulus and otherwise be segregated. To address the challenge of recognizing whether sensory cues come from the same or different stimuli that are unknown in advance, we propose that the brain should carry out multisensory integration and segregation concurrently with two different neuron groups. Specifically, congruent neurons implement cue integration, while opposite neurons compute the cue disparity, and the interplay between them achieves rapid stimulus recognition without information loss. We apply our model to the example of inferring heading-direction based on visual and vestibular cues and reproduce the experimental data successfully.

Reciprocally Coupled Local Estimators Implement Bayesian Information Integration Distributively.

Decentralized Multisensory Information Integration in Neural Systems

Concurrent Multisensory Integration and Segregation with Complementary Congruent and Opposite Neurons

Simultaneously Optimizing Spatial Spectral Features Based on Mutual Information for Eeg Classification

Decentralized Neural Circuits of Multisensory Information Integration in the Brain

Antagonistic Cross-Links and Reciprocal Couplings Emerge in Optimal Multisensory Integration

Unveiling Dynamic System Strategies for Multisensory Processing: from Neuronal Fixed-Criterion Integration to Population Bayesian Inference

How the Prior Information Shapes Neural Networks for Optimal Multisensory Integration

Complementary Congruent and Opposite Neurons Achieve Concurrent Multisensory Integration and Segregation.

Development of a Bayesian Estimator for Audio-Visual Integration: A Neurocomputational Study

Encoding Multisensory Information In Modular Neural Networks

Visual-Based Spatial Coordinate Dominates Probabilistic Multisensory Inference in Macaque MST-d Disparity Encoding

To integrate or not to integrate: Temporal dynamics of hierarchical Bayesian causal inference

Multisensory integration across reference frames with additive feed-forward networks

Neural Correlates of Optimal Multisensory Decision Making under Time-Varying Reliabilities with an Invariant Linear Probabilistic Population Code

Interhemispheric Multisensory Perception and Bayesian Causal Inference.

Continuous attractor neural network model of multisensory integration

Author Response for "single Stimulus Location for Two Inputs: A Combined Brain‐computer Interface Based on Steady State Visual Evoked Potential (SSVEP)"

Distributed Sampling-based Bayesian Inference in Coupled Neural Circuits

Closing the loop on multisensory interactions: A neural architecture for multisensory causal inference and recalibration

The Cognitive and Neural Mechanism of Multisensory Information Integrating:A Review of Current Studies