Abstract:Social interaction is a common phenomenon in human societies. Different from discovering groups based on the similarity of individuals’ actions, social interaction focuses more on the mutual influence between people. Although people can easily judge whether or not there are social interactions in a real-world scene, it is difficult for an intelligent system to discover social interactions. Initiating and concluding social interactions are greatly influenced by an individual’s social cognition and the surrounding environment, which are closely related to psychology. Thus, converting the psychological factors that impact social interactions into quantifiable visual representations and creating a model for interaction relationships poses a significant challenge. To this end, we propose a Psychology-Guided Environment Aware Network (PEAN) that models social interaction among people in videos using supervised learning. Specifically, we divide the surrounding environment into scene-aware visual-based and human-aware visual-based descriptions. For the scene-aware visual clue, we utilize 3D features as global visual representations. For the human-aware visual clue, we consider instance-based location and behaviour-related visual representations to map human-centered interaction elements in social psychology: distance, openness and orientation. In addition, we design an environment aware mechanism to integrate features from visual clues, with a Transformer to explore the relation between individuals and construct pairwise interaction strength features. The interaction intensity matrix reflecting the mutual nature of the interaction is obtained by processing the interaction strength features with the interaction discovery module. An interaction constrained loss function composed of interaction critical loss function and smooth F β loss function is proposed to optimize the whole framework to improve the distinction of the interaction matrix and alleviate class imbalance caused by pairwise interaction sparsity. Given the diversity of real-world interactions, we collect a new dataset named Social Basketball Activity Dataset (Soical-BAD), covering complex social interactions. Our method achieves the best performance among social-CAD, social-BAD, and their combined dataset named Video Social Interaction Dataset (VSID).

Psychology-Inspired Interaction Process Analysis based on Time Series.

Real-Time Eye-Gaze Based Interaction for Human Intention Prediction and Emotion Analysis.

Interpersonal Relationship Analysis with Dyadic EEG Signals via Learning Spatial-Temporal Patterns

Analyzing Temporal Dynamics of Dyadic Synchrony in Affective Interactions.

Evaluating Temporal Patterns in Applied Infant Affect Recognition

Computational Approaches to Understanding Interaction and Development

Clarifying the Interaction Types in Two-Person Neuroscience Research.

Computational Social Dynamics: Analyzing the Face-level Interactions in a Group

Affective Interaction Recognition Using Spatio-Temporal Features and Context

CogIntAc: Modeling the Relationships between Intention, Emotion and Action in Interactive Process from Cognitive Perspective

Psychology-Guided Environment Aware Network for Discovering Social Interaction Groups from Videos

Affect Control Processes: Intelligent Affective Interaction using a Partially Observable Markov Decision Process

Perception of Human Interaction Based on Motion Trajectories: From Aerial Videos to Decontextualized Animations.

Toward Improving Social Interaction Ability for Children with Autism Spectrum Disorder Using Social Signals

Psychodynamic Appraisal Mechanism for Emotional Development Through Multi-modal Interaction

Temporal Interaction -- Bridging Time and Experience in Human-Computer Interaction

The Dynamics of Triple Interactions in Resting fMRI: Insights into Psychotic Disorders

An advanced mental state transition network and psychological experiments

Towards Improving Social Communication Skills with Multimodal Sensory Information

Predicting Trust Using Automated Assessment of Multivariate Interactional Synchrony

Dynamic Patterns of Affect-Biased Attention in Children and Its Relationship with Parenting