Abstract:The quantity of music content is rapidly increasing and automated affective tagging of music video clips can enable the development of intelligent retrieval, music recommendation, automatic playlist generators, and music browsing interfaces tuned to the users' current desires, preferences, or affective states. To achieve this goal, the field of affective computing has emerged, in particular the development of so-called affective brain-computer interfaces, which measure the user's affective state directly from measured brain waves using non-invasive tools, such as electroencephalography (EEG). Typically, conventional features extracted from the EEG signal have been used, such as frequency subband powers and/or inter-hemispheric power asymmetry indices. More recently, the coupling between EEG and peripheral physiological signals, such as the galvanic skin response (GSR), have also been proposed. Here, we show the importance of EEG amplitude modulations and propose several new features that measure the amplitude-amplitude cross-frequency coupling per EEG electrode, as well as linear and non-linear connections between multiple electrode pairs. When tested on a publicly available dataset of music video clips tagged with subjective affective ratings, support vector classifiers trained on the proposed features were shown to outperform those trained on conventional benchmark EEG features by as much as 6, 20, 8, and 7% for arousal, valence, dominance and liking, respectively. Moreover, fusion of the proposed features with EEG-GSR coupling features showed to be particularly useful for arousal (feature-level fusion) and liking (decision-level fusion) prediction. Together, these findings show the importance of the proposed features to characterize human affective states during music clip watching.

Multimedia content analysis for emotional characterization of music video clips

EmoPlayer: A Media Player for Video Clips with Affective Annotations.

Exploiting evidential theory in the fusion of textual, audio, and visual modalities for affective music video retrieval

A color bar based affective annotation method for media player

Music video affective understanding using feature importance analysis

Electroencephalography Amplitude Modulation Analysis for Automated Affective Tagging of Music Video Clips

Affective Visualization and Retrieval for Music Video

Exploiting EEG signals and audiovisual feature fusion for video emotion recognition

Identifying affective levels on music video via completing the missing modality

Impact of Affective Multimedia Content on the Electroencephalogram and Facial Expressions

Study on Linguistic Computing for Music Emotion

Predicting emotion from music videos: exploring the relative contribution of visual and auditory information to affective responses

Implicit Media Tagging and Affect Prediction from video of spontaneous facial expressions, recorded with depth camera

An Affective Video Database using Multimedia Content Analysis rated on Indian samples

Video indexing and recommendation based on affective analysis of viewers.

Correlation-Based Feature Selection And Regression

Video affective content analysis: a survey of state of the art methods

Affective MTV analysis based on arousal and valence features

Hybrid video emotional tagging using users’ EEG and video content

Towards an intelligent framework for multimodal affective data analysis

Inter-Brain EEG Feature Extraction and Analysis for Continuous Implicit Emotion Tagging During Video Watching