Abstract:In this paper, we present a multimodal dataset for affective computing research acquired in a human-computer interaction (HCI) setting. An experimental mobile and interactive scenario was designed and implemented based on a gamified generic paradigm for the induction of dialog-based HCI relevant emotional and cognitive load states. It consists of six experimental sequences, inducing Interest, Overload, Normal, Easy, Underload, and Frustration. Each sequence is followed by subjective feedbacks to validate the induction, a respiration baseline to level off the physiological reactions, and a summary of results. Further, prior to the experiment, three questionnaires related to emotion regulation (ERQ), emotional control (TEIQue-SF), and personality traits (TIPI) were collected from each subject to evaluate the stability of the induction paradigm. Based on this HCI scenario, the University of Ulm Multimodal Affective Corpus (uulmMAC), consisting of two homogenous samples of 60 participants and 100 recording sessions was generated. We recorded 16 sensor modalities including 4 × video, 3 × audio, and 7 × biophysiological, depth, and pose streams. Further, additional labels and annotations were also collected. After recording, all data were post-processed and checked for technical and signal quality, resulting in the final uulmMAC dataset of 57 subjects and 95 recording sessions. The evaluation of the reported subjective feedbacks shows significant differences between the sequences, well consistent with the induced states, and the analysis of the questionnaires shows stable results. In summary, our uulmMAC database is a valuable contribution for the field of affective computing and multimodal data analysis: Acquired in a mobile interactive scenario close to real HCI, it consists of a large number of subjects and allows transtemporal investigations. Validated via subjective feedbacks and checked for quality issues, it can be used for affective computing and machine learning applications.

AMIGOS: A Dataset for Affect, Personality and Mood Research on Individuals and Groups

DEAP: A Database for Emotion Analysis ;Using Physiological Signals

AVDOS-VR: Affective Video Database with Physiological Signals and Continuous Ratings Collected Remotely in VR

Emo-FilM: A multimodal dataset for affective neuroscience using naturalistic stimuli

The OMG-Empathy Dataset: Evaluating the Impact of Affective Behavior in Storytelling

DynAMoS: The Dynamic Affective Movie Clip Database for Subjectivity Analysis

"Biovid Emo DB": A Multimodal Database for Emotion Analyses Validated by Subjective Ratings.

SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild

The uulmMAC Database—A Multimodal Affective Corpus for Affective Computing in Human-Computer Interaction

A dataset of continuous affect annotations and physiological signals for emotion analysis

An Affective Video Database using Multimedia Content Analysis rated on Indian samples

MDEAW: A Multimodal Dataset for Emotion Analysis through EDA and PPG signals from wireless wearable low-cost off-the-shelf Devices

Multimodal Spontaneous Emotion Corpus for Human Behavior Analysis

Audio-Visual Automatic Group Affect Analysis

A real-world dataset of group emotion experiences based on physiological data

WEMAC: Women and Emotion Multi-modal Affective Computing dataset

MPED: A Multi-Modal Physiological Emotion Database for Discrete Emotion Recognition

DREAMER: A Database for Emotion Recognition Through EEG and ECG Signals From Wireless Low-cost Off-the-Shelf Devices

The design and evaluation of multimode affective database

Multimodal Affective State Assessment Using fNIRS + EEG and Spontaneous Facial Expression

The CP-QAE-I: A Video Dataset for Exploring the Effect of Personality and Culture on Perceived Quality and Affect in Multimedia