Abstract:Collaborating in a group, whether face-to-face or virtually, involves continuously expressing emotions and interpreting those of other group members. Therefore, understanding group affect is essential to comprehending how groups interact and succeed in collaborative efforts. In this study, we move beyond individual-level affect and investigate group-level affect -- a collective phenomenon that reflects the shared mood or emotions among group members at a particular moment. As the first in literature, we gather annotations for group-level affective expressions using a fine-grained temporal approach (15 second windows) that also captures the inherent dynamics of the collective construct. To this end, we use trained annotators and an annotation procedure specifically tuned to capture the entire scope of the group interaction. In addition, we model group affect dynamics over time. One way to study the ebb and flow of group affect in group interactions is to model the underlying convergence (driven by emotional contagion) and divergence (resulting from emotional reactivity) of affective expressions amongst group members. To capture these interpersonal dynamics, we extract synchrony based features from both audio and visual social signal cues. An analysis of these features reveals that interacting groups tend to diverge in terms of their social signals along neutral levels of group affect, and converge along extreme levels of affect expression. We further present results on the predictive modeling of dynamic group affect which underscores the importance of using synchrony-based features in the modeling process, as well as the multimodal nature of group affect. We anticipate that the presented models will serve as the baselines of future research on the automatic recognition of dynamic group affect.

Audio-Visual Automatic Group Affect Analysis

Video affective content analysis: a survey of state of the art methods

Affective Video Content Analysis: Decade Review and New Perspectives

An Affective Video Database using Multimedia Content Analysis rated on Indian samples

A Novel Affective Visualization System for Videos Based on Acoustic and Visual Features

Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework

Affective Behaviour Analysis via Integrating Multi-Modal Knowledge

Implicit Media Tagging and Affect Prediction from video of spontaneous facial expressions, recorded with depth camera

Dynamics of Collective Group Affect: Group-level Annotations and the Multimodal Modeling of Convergence and Divergence

Affective Video Content Analysis Based on Two Compact Audio-Visual Features.

Affective Video Content Analysis Based on Multimodal Data Fusion in Heterogeneous Networks.

An Ensemble Approach for Facial Expression Analysis in Video

Global Affective Video Content Regression Based on Complementary Audio-Visual Features

AMIGOS: A Dataset for Affect, Personality and Mood Research on Individuals and Groups

Beyond Facial Expressions: Learning Human Emotion from Body Gestures.

Exploring Multimodal Visual Features For Continuous Affect Recognition

VEATIC: Video-based Emotion and Affect Tracking in Context Dataset

Multimodal Sentiment Intensity Analysis in Videos: Facial Gestures and Verbal Messages

Impact of Affective Multimedia Content on the Electroencephalogram and Facial Expressions

Towards Intercultural Affect Recognition: Audio-Visual Affect Recognition in the Wild Across Six Cultures

An Effective Ensemble Learning Framework for Affective Behaviour Analysis