Abstract:With the rise of social media as a primary channel for information exchange, the spread of online public sentiment has become a key factor in escalating social conflicts and triggering public concern, exerting profound influences on social stability and public values. On some instant interaction platforms, social media texts are often short and noisy, rendering traditional sentiment recognition methods unsuitable for scenarios with scarce short-text content and semantic and emotional uncertainty. To address these challenges, this study proposes a TVE-MGF model (Textual-Visual Context Enhancement and Multi-Granularity Semantic Fusion Model), which utilizes ViLBERT for indepth multimodal feature extraction from both visual context and semantic content. In addition, by enhancing the expression of textual and visual semantics, the model achieves multi-granularity fusion of social media text and associated images, enhancing the precision and comprehensive understanding in sentiment analysis. Furthermore, we adopted feature distillation techniques to optimize the TVE-MGF model, aiming to more effectively extract and utilize implicit knowledge from both visual and textual data to construct higher-level semantic feature representations. This step has bolstered the model’s capacity for generalization and extraction of critical knowledge, significantly improving performance when handling complex multimodal sentiment data. Finally, the methodology was experimentally validated on two multimodal datasets, MVSA-single and MVSA-multiple, with the TVE-MGF model achieving F1 scores of $78.10 \%$ and $79.03 \%$ respectively, thereby demonstrating its effectiveness in enhancing the efficiency of sentiment recognition in social media, particularly for texts with high semantic and emotional uncertainty.

Image Sentiment Analysis Based on Collective Matrix Factorization

IMAGE SENTIMENT ANALYSIS USING SUPERVISED COLLECTIVE MATRIX FACTORIZATION

Flickr Image Community Analytics by Deep Noise-Refined Matrix Factorization.

FGCM: Noisy Label Learning via Fine-Grained Confidence Modeling

Cauchy Matrix Factorization for Tag-Based Social Image Retrieval.

Social Image Sentiment Analysis by Exploiting Multimodal Content and Heterogeneous Relations

Social Image-text Sentiment Classification With Cross-Modal Consistency and Knowledge Distillation

Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks

Image–Text Sentiment Analysis Via Context Guided Adaptive Fine-Tuning Transformer

Transfer Correlation Between Textual Content to Images for Sentiment Analysis

Sentiment Analysis for Images on Microblogging by Integrating Textual Information with Multiple Kernel Learning.

Visual Sentiment Analysis Using Deep Learning Models with Social Media Data

Non-Linear Matrix Completion for Social Image Tagging.

Nonnegative Matrix Factorization for Multimodality Data from Multi-source Domain

Cross-modal image sentiment analysis via deep correlation of textual semantic

Image and Text Aspect Level Multimodal Sentiment Classification Model Using Transformer and Multilayer Attention Interaction

Multimodal sentiment analysis based on cross-instance graph neural networks

DGFN Multimodal Emotion Analysis Model Based on Dynamic Graph Fusion Network

Context-aware Image Tweet Modelling and Recommendation.

Cross-Domain Feature Learning in Multimedia

Multi-modal Feature Fistillation Emotion Recognition Method for Social Media