Abstract:Abstract Affective computing has suffered by the precise annotation because the emotions are highly subjective and vague. The music video emotion is complex due to the diverse textual, acoustic, and visual information which can take the form of lyrics, singer voice, sounds from the different instruments, and visual representations. This can be one reason why there has been a limited study in this domain and no standard dataset has been produced before now. In this study, we proposed an unsupervised method for music video emotion analysis using music video contents on the Internet. We also produced a labelled dataset and compared the supervised and unsupervised methods for emotion classification. The music and video information are processed through a multimodal architecture with audio–video information exchange and boosting method. The general 2D and 3D convolution networks compared with the slow–fast network with filter and channel separable convolution in multimodal architecture. Several supervised and unsupervised networks were trained in an end-to-end manner and results were evaluated using various evaluation metrics. The proposed method used a large dataset for unsupervised emotion classification and interpreted the results quantitatively and qualitatively in the music video that had never been applied in the past. The result shows a large increment in classification score using unsupervised features and information sharing techniques on audio and video network. Our best classifier attained 77% accuracy, an f1-score of 0.77, and an area under the curve score of 0.94 with minimum computational cost.

Affective Classification In Video Based On Semi-Supervised Learning

Affective Video Classification Based on Spatio-temporal Feature Fusion

Exploiting EEG signals and audiovisual feature fusion for video emotion recognition

Semi-Supervised Classification of Emotional Pictures Based on Feature Combination

Music video emotion classification using slow–fast audio–video network and unsupervised feature representation

Video Classification and Recommendation Based on Affective Analysis of Viewers

Affective image classification via semi-supervised learning from web images

Affective image classification by jointly using interpretable art features and semantic annotations

Affective Video Content Analysis Based on Multimodal Data Fusion in Heterogeneous Networks.

Video affective content recognition based on unascertained measure

Study on unascertained clustering for video affective recognition

Affective Image Classification by Jointly Using Low-Level Visual Features and Interpretable Aesthetic Features

Video affective content analysis: a survey of state of the art methods

Synchronous Prediction of Arousal and Valence Using LSTM Network for Affective Video Content Analysis

A Video Retrieval Algorithm Based on Affective Features

Multi-modal Learning for Affective Content Analysis in Movies

Semi-supervised emotional classification of color images by learning from cloud

Visual Affective Classification by Combining Visual and Text Features

Film Affective Content Recognition Based on Fuzzy Inference

A Novel Affective Visualization System for Videos Based on Acoustic and Visual Features

Hybrid video emotional tagging using users’ EEG and video content