Contribution of auditory distance cues to size constancy in perception and grasping when visual cues are compromised

Chao Zheng,Gexiu Wang,Juan Chen

DOI: https://doi.org/10.1167/jov.22.14.3870

2022-01-01

Journal of Vision

Abstract:People need to integrate information from multiple modalities to obtain precise information about object features and spatial relationships. An extreme case for multiple sensory integrations is that when information from one source is compromised, information from other sources would have a more significant weight in the integration to compensate for the loss. Although previous studies have shown that the weighting of sensory information depends on the reliability of that information, our previous study with visual and proprioceptive information indicates that the weighting also depends on the nature of the control system (visual perceptual or visuomotor systems) employed in the tasks. Here we tested whether or not this is also the case for visual-auditory integration. We tested size constancy in perception and grasping when there were ample visual (i.e., viewing binocularly in light), restricted visual (i.e., viewed monocularly through a 1 mm hole in entirely dark), and restricted visual but some auditory distance cues. In the ample-vision condition, participants showed size constancy in both tasks. In the restricted-vision condition, size constancy was disrupted in both tasks. Interestingly, when auditory distance cues were provided in this situation, only size constancy in perception, but not in grasping, was improved, suggesting that the visual perceptual system can better use auditory information than the visuomotor system. We then trained participants to use the auditory signal to discriminate distances in seven days and tested size constancy afterward. After training, participants’ performance in distance discrimination and the size constancy in grasping were all improved, suggesting that the more reliable a cue, the more significant contribution it is in sensory integration. Taken together, we provide additional evidence that both the nature of the control system, the tasks employed, and the reliability of the cue influence the weighting of cues in multisensory integration.