Abstract:With the continuous development of the research in the field of emotion analysis, music, as a common multimodal information carrier in people's daily life, often transmits emotion through lyrics and melody, so it has been gradually incorporated into the research category of emotion analysis. The fusion classification model based on CNN-LSTM proposed in this paper effectively improves the accuracy of emotional classification of audio and lyrics. At the same time, in view of the problem that the traditional decision-level fusion method ignores the correlation between modes and the limitations of dataset, this paper further improves the existing Thayer dimension emotional decision fusion method, takes the audio energy axis data as the main discrimination basis, and improves the accuracy of decision fusion classification. Based on the results of music emotion analysis, this paper further carries out the task of music generation. Based on the feature that there is often consistent emotional expression between music words and songs, a dual Seq2Seq framework based on reinforcement learning is constructed. By introducing the reward value of emotional consistency and content fidelity, the output melody has the same emotion with the input lyrics and good results are achieved. Compared with the ordinary Seq2Seq, the accuracy of our proposed model is improved by about 1.1%. This shows that the accuracy of the model can be effectively improved by using reinforcement learning.

Automatic Music Mood Classification by Learning Cross-Media Relevance Between Audio and Lyrics

Automatic Music Emotion Classification Using a New Classification Algorithm

Graph-Based Multimodal Music Mood Classification in Discriminative Latent Space.

Multimodal Music Mood Classification by Fusion of Audio and Lyrics.

Mind Band: A Crossmedia AI Music Composing Platform

Music Mood Classification Based On Lifelog

Improve the application of reinforcement learning and multi‐modal information in music sentiment analysis

Enhancing Music Mood Recognition with LLMs and Audio Signal Processing: A Multimodal Approach

Mood Classification of Music Using AdaBoost

Real-Time Human-Music Emotional Interaction Based on Deep Learning and Multimodal Sentiment Analysis

Multi-Modal Music Mood Classification Using Co-Training

Boosting for Multi-Modal Music Emotion Classification.

Enriching Music Mood Annotation By Semantic Association Reasoning

An artificial intelligence-based classifier for musical emotion expression in media education

Real-time Human-Music Emotional Interaction Based on Multimodal Analysis

User-Adaptive Music Emotion Recognition

Digital Empirical Research of Influencing Factors of Musical Emotion Classification Based on Pleasure-Arousal Musical Emotion Fuzzy Model

Joint sentiment analysis of lyrics and audio in music

Research on Music Emotional Expression Based on Reinforcement Learning and Multimodal Information

Deep learning based mood tagging for Chinese song lyrics

Music Emotion Research Based on Reinforcement Learning and Multimodal Information