Abstract:The affect embedded in video data conveys high-level semantic information about the content and has direct impact on the understanding and perception of reviewers, as well as their emotional responses. Affective Video Content Analysis (AVCA) attempts to generate a direct mapping between video content and the corresponding affective states such as arousal and valence dimensions. Most existing studies establish the mapping for each dimension separately using knowledge-based rules or traditional classifiers such as Support Vector Machine (SVM). The inherent correlations between affective dimensions have largely been unexploited, which are anticipated to include important information for accurate prediction of affective dimensions. To address this issue, this paper presents an approach to predict arousal and valance dimensions synchronously using the Long Short Term Memory (LSTM) network. The approach extracts a set of low-level audio and visual features from video data and projects them synchronously into pairs of arousal and valence values using the LSTM network which automatically incorporates the correlations between arousal and valance dimensions. We evaluate the performance of the proposed approach on a dataset comprising video clips segmented from real-world resources such as film, drama, and news, and demonstrate its superior performance over the traditional SVM based method. The results provide one of the earliest preliminary evidence to the benefit of considering correlations between affective dimensions towards accurate AVCA.

A Locally Weighted Method To Improve Linear Regression For Lexical-Based Valence-Arousal Prediction

A Regression Approach to Valence-Arousal Ratings of Words from Word Embedding.

Valence-arousal Ratings Prediction with Co-Occurrence Word-Embedding

Inferring Affective Meanings of Words from Word Embedding

Chinese EmoBank: Building Valence-Arousal Resources for Dimensional Sentiment Analysis

Affective awareness in neural sentiment analysis

Synchronous Prediction of Arousal and Valence Using LSTM Network for Affective Video Content Analysis

An Intelligent Logistic Regression Approach for Verb Expression’s Sentiment Analysis

Automatic Construction and Optimization of Sentiment Lexicon Based on Word2Vec

Visual Sentiment Prediction Based on Automatic Discovery of Affective Regions

Dimensional Sentiment Analysis of Traditional Chinese Words Using Pre-Trained Not-Quite-Right Sentiment Word Vectors and Supervised Ensemble Models

Visual Sentiment Analysis by Leveraging Local Regions and Human Faces.

Exploring English Lexicon Knowledge for Chinese Sentiment Analysis.

A LDA-based approach to lyric emotion regression

Using Support Vector Regression To Estimate Valence Level From Eeg

Affect Estimation in 3D Space Using Multi-Task Active Learning for Regression

Sentiment Analysis with Global Topics and Local Dependency

Automatic Construction and Global Optimization of a Multisentiment Lexicon.

Effective Vietnamese Sentiment Analysis Model Using Sentiment Word Embedding and Transfer Learning

Emotional Variance Analysis: A new sentiment analysis feature set for Artificial Intelligence and Machine Learning applications

Using Bilingual Lexicon to Judge Sentiment Orientation of Chinese Words