Abstract:A mere co-presence of an unfamiliar person may modulate an individual's attentive engagement with specific events or situations to a significant degree. To understand better how such social presence affects experiences, we recorded a set of parallel multimodal facial and psychophysiological data with subjects (N = 36) who listened to dramatic audio scenes alone or when facing an unfamiliar person. Both a selection of 6 s affective sound clips (IADS-2) followed by a 27 min soundtrack extracted from a Finnish episode film depicted familiar and often intense social situations familiar from the everyday world. Considering the systemic complexity of both the chosen naturalistic stimuli and expected variations in the experimental social situation, we applied a novel combination of signal analysis methods using inter-subject correlation (ISC) analysis, Representational Similarity Analysis (RSA) and Recurrence Quantification Analysis (RQA) followed by gradient boosting classification. We report our findings concerning three facial signals, gaze, eyebrow and smile that can be linked to socially motivated facial movements. We found that ISC values of pairs, whether calculated on true pairs or any two individuals who had a partner, were lower than the group with single individuals. Thus, audio stimuli induced more unique responses in those subjects who were listening to it in the presence of another person, while individual listeners tended to yield a more uniform response as it was driven by dramatized audio stimulus alone. Furthermore, our classifiers models trained using recurrence properties of gaze, eyebrows and smile signals demonstrated distinctive differences in the recurrence dynamics of signals from paired subjects and revealed the impact of individual differences on the latter. We showed that the presence of an unfamiliar co-listener that modifies social dynamics of dyadic listening tasks can be detected reliably from visible facial modalities. By applying our analysis framework to a broader range of psycho-physiological data, together with annotations of the content, and subjective reports of participants, we expected more detailed dyadic dependencies to be revealed. Our work contributes towards modeling and predicting human social behaviors to specific types of audio-visually mediated, virtual, and live social situations.

Reduction in Ethanol Self-Selection of C57BL/6j Mice During Treatment with 3-((2-Imidazoline-2yl) Methyl) Indole

Asymmetrical Cross-Modal Influence on Neural Encoding of Auditory and Visual Features in Natural Scenes.

Virtually spatialized sounds enhance auditory processing in healthy participants and patients with a disorder of consciousness

Further studies on the inactivator of glucosamine 6-phosphate synthase in rat submaxillary gland. Identification as an esteroprotease.

A Quality of Experience and Visual Attention Evaluation for 360° videos with non-spatial and spatial audio

Effect of acoustic scene complexity and visual scene representation on auditory perception in virtual audio-visual environments

Interaction effect of reproducing spatialized human-induced and environmental sounds from nearby sources on the social Simon effect

Assessing Behavioral and Neural Correlates of Change Detection in Spatialized Acoustic Scenes

Electroencephalographic Correlates of Sensorimotor Integration and Embodiment during the Appreciation of Virtual Architectural Environments

Predictors of the Sense of Presence in an Immersive Audio Storytelling Experience, a Mixed Methods Study. PREPRINT

Presence, flow, and narrative absorption: an interdisciplinary theoretical exploration with a new spatiotemporal integrated model based on predictive processing

The Blood and Bone Marrow in Yelloe Cross Gas (Mustard Gas) Poisoning: Changes produced in the Bone Marrow of Fatal Cases.

Using electrophysiological measures to evaluate the sense of presence in immersive virtual environments: An event‐related potential study

Sensory and Perceptual Decisional Processes Underlying the Perception of Reverberant Auditory Environments

Active listening modulates the spatial hearing experience: a multicentric study

Concurrent talking in immersive virtual reality: on the dominance of visual speech cues

Modulation of Brain Activity by Selective Attention to Audiovisual Dialogues

Retinotopic effects during spatial audio-visual integration

Gas gangrene: Report of an unusual case, with a therapeutic program for gas gangrene

Unraveling dyadic psycho-physiology of social presence between strangers during an audio drama - a signal-analysis approach

Attention Modulates the Auditory Cortical Processing of Spatial and Category Cues in Naturalistic Auditory Scenes