Abstract:Digital video is widely used to record people's daily lives and share people's moods, but few researchers have conducted research on the consistency of emotional expression between short videos and music. In order to be able to match the appropriate background music to the short video image autonomously and efficiently, the paper analyzed the emotional connection between the two from the audio-visual synesthesia. First, emotional semantics was used as a bridge to connect video data and music data, and a video-music synesthesia data set based on semantic words was constructed. Then, an attention mechanism was incorporated to better extract key features in video images. In the extraction of music features, an improved lenet5 network was used, and the optimal network parameters were determined through experiments. Finally, the two types of features were fused and the mutual retrieval between video and music was performed. In order to compare the performance of different models, different CNN models were calculated in the processing of video images, including VGG16, VGG19, AlexNet and GoogleNet, and the attention mechanism was added to each network for calculation to compare its retrieval accuracy. In the processing of music data, different CNN algorithms were also used for comparative experiments, and networks with different layers were used to determine the optimal results. The experimental results show that the audiovisual synesthesia retrieval model based on emotion can effectively measure the emotional similarity between video images and music, and the method of the paper can produce a good match between them. The research method of the paper is the exploration of computer synesthetic intelligence, which can stimulate the creative inspiration of image and music creative designers. While enhancing the emotional experience of digital products, it also improves the efficiency and quality of development.

Fine-grained Sentiment Semantic Analysis and Matching of Music and Image

Emotion-Driven Chinese Folk Music-Image Retrieval Based on De-Svm

Image–Music Synesthesia-Aware Learning Based on Emotional Similarity Recognition

Human-centric Music Medical Therapy Exploration System

Automatic Music Emotion Classification Using a New Classification Algorithm

Learning Music Emotion Primitives via Supervised Dynamic Clustering.

Linguistic computing model for music emotion

Study on Linguistic Computing for Music Emotion

Emotion-Based End-to-End Matching Between Image and Music in Valence-Arousal Space

An Audiovisual Correlation Matching Method Based on Fine-Grained Emotion and Feature Fusion

Music's Affective Computing Model Based on Fuzzy Logic

Research on music emotion cognition model based on linguistic value computing

Emotion Based Image Musicalization

Digital Empirical Research of Influencing Factors of Musical Emotion Classification Based on Pleasure-Arousal Musical Emotion Fuzzy Model

Exploring and Applying Audio-Based Sentiment Analysis in Music

Real-Time Human-Music Emotional Interaction Based on Deep Learning and Multimodal Sentiment Analysis

Design of Semantic Matching Model of Folk Music in Occupational Therapy Based on Audio Emotion Analysis

Research on Emotional Semantic Retrieval of Attention Mechanism Oriented to Audio-visual Synesthesia

Real-time Human-Music Emotional Interaction Based on Multimodal Analysis

Intelligent Analysis Of Music'S Affective Features And Fxpressive Pattern

Intelligent Analysis of Music's Affective Features and Expressive Pattern