Abstract:Human interactions contain potent social cues that not only meet the eye but also the ear. Although research has identified a region in the posterior superior temporal sulcus as being particularly sensitive to visually presented social interactions (SI-pSTS), its response to auditory interactions has not been tested. Here, we used fMRI to explore brain response to auditory interactions, with a focus on temporal regions known to be important in auditory processing and social interaction perception. In Experiment 1, monolingual participants listened to two-speaker conversations (intact or sentence-scrambled) and one-speaker narrations in both a known and unknown language. Speaker number and conversational coherence were explored in separately localised regions-of-interest (ROI). In Experiment 2, bilingual participants were scanned to explore the role of language comprehension. Combining univariate and multivariate analyses, we found initial evidence for a heteromodal response to social interactions in SI-pSTS. Specifically, right SI-pSTS preferred auditory interactions over control stimuli and represented information about both speaker number and interactive coherence. Bilateral temporal voice areas (TVA) showed a similar, but less specific, profile. Exploratory analyses identified another auditory-interaction sensitive area in anterior STS. Indeed, direct comparison suggests modality specific tuning, with SI-pSTS preferring visual information while aSTS prefers auditory information. Altogether, these results suggest that right SI-pSTS is a heteromodal region that represents information about social interactions in both visual and auditory domains. Future work is needed to clarify the roles of TVA and aSTS in auditory interaction perception and further probe right SI-pSTS interaction-selectivity using non-semantic prosodic cues. Highlights - Novel work investigating social interaction perception in the auditory domain. - Visually defined SI-pSTS shows a heteromodal response profile to interactions. - Yet, it prefers visual to auditory stimuli. The reverse was found for anterior STS. - Temporal voice areas show qualitatively different response compared to SI-pSTS. - Future studies are needed to corroborate the unique role of right SI-pSTS.

Interaction effect of reproducing spatialized human-induced and environmental sounds from nearby sources on the social Simon effect

Effect of acoustic scene complexity and visual scene representation on auditory perception in virtual audio-visual environments

How Does Visual Context of Urban Open Spaces Drive Soundscape Pleasantness and Eventfulness: a Laboratory Experiment Using a Higher Order Ambisonics Speaker System and Immersive Virtual Reality

The impact of binaural auralizations on sound source localization and social presence in audiovisual virtual reality: converging evidence from placement and eye-tracking paradigmsâ ̃

An Experimental Study on the Role of Software Synthesized 3D Sound in Augmented Reality Environments

From 3D landscape visualization to environmental simulation: The contribution of sound to the perception of virtual environments

Spatial audio reproduction for studying second language speech perception in varying acoustic environments

An Experimental Study on the Role of 3D Sound in Augmented Reality Environment.

The Role of 3-D Sound in Human Reaction and Performance in Augmented Reality Environments

Auditory dyadic interactions through the “eye” of the social brain: How visual is the posterior STS interaction region?

The effect of Surround sound on embodiment and sense of presence in cinematic experience: a behavioral and HD-EEG study

Where is the Sound: User Sound Source Perception in Virtual Reality Environment

On the Use of Multi-sensory Cues in Symmetric and Asymmetric Shared Collaborative Virtual Spaces

Effect of Human Sound Component on the Sociability of Urban Public Spaces—a Case Study in Sheffield, UK

Acoustic scene complexity affects motion behavior during speech perception in audio-visual multi-talker virtual environments

Comparison of subjective evaluations in virtual and real environments for soundscape researcha)

Perception Difference for Approaching and Receding Sound Sources of a Listener in Motion in Architectural Sequential Spaces

Evaluating the perceived affective qualities of urban soundscapes through audiovisual experiments

The Influence of Companion Factors on Soundscape Evaluations in Urban Public Spaces

Concurrent talking in immersive virtual reality: on the dominance of visual speech cues

Ecological Validity of Immersive Virtual Reality (IVR) Techniques for the Perception of Urban Sound Environments