Abstract:<p>Social media has become indispensable to people's lives, where they can share their views and emotion with images and texts. Analyzing social images for sentiment prediction can help understand human social behavior and provide better recommendation results. Most current researches on image sentiment analysis have achieved quite good progress, which ignores the semantic correlation between an image and its corresponding descriptive sentences (caption). To capture the complementary multimodal information for joint sentiment classification, in this paper, we propose a novel cross-modal Semantic Content Correlation(SCC) method based on deep matching and hierarchical networks, which bridges the correlation between images and captions. Specifically, pre-trained convolutional neural networks (CNNs) are leveraged to encode the visual sub-regions contents, and a GloVe is employed to embed the textual semantic. Relying on visual contents and textual semantic, a joint attention network is proposed to learn the content correlation of the image and its caption, which is then exported as an image-text pair. To exploit the dependence of visual contents on textual semantic in caption effectively, the caption is processed by a Class-Aware Sentence Representation (CASR) network with a class dictionary, and a fully connected layer concatenates the outputs of CASR into a class-aware vector. Finally, the class-aware distributed vector is fed into an Inner-class Dependency Long Short-Term Memory network (IDLSTM) with the image–text pair as a query to further capture the cross-modal non-linear correlations for sentiment prediction. The performance of extensive experiments conducted on three datasets verifies the effectiveness of the model SCC.</p>

Correlation Identification in Multimodal Weibo Via Back Propagation Neural Network with Genetic Algorithm

Research on Weibo Public Opinion Prediction Using Improved Genetic Algorithm Based BP Neural Networks

A Fast Prediction Algorithm for Sina Weibo Users with Time Correlation Cognition

A Deep Approach For Multi-Modal User Attribute Modeling

DGHC: A Hybrid Algorithm for Multi-Modal Named Entity Recognition Using Dynamic Gating and Correlation Coefficients With Visual Enhancements

Cross-Modality Microblog Sentiment Prediction Via Bi-Layer Multimodal Hypergraph Learning

Weibo Spammer Detection Based On Social Network Digital Twin.

User-based Hierarchical Network of Sina Weibo Emotion Analysis

Recognition of College Students from Weibo with Deep Neural Networks

The Influence of Digitization on the Dissemination of Traditional Chinese Music and Weibo Content Propagation Under Deep Learning

Multimodal sentiment analysis based on cross-instance graph neural networks

Multimodal correlation deep belief networks for multi-view classification

Multi-modal microblog classification via multi-task learning

Cross-modal image sentiment analysis via deep correlation of textual semantic

Identifying Important Users in Sina Microblog

Multi-label Emotion Classification for Tweets in Weibo: Method and Application

Event Classification in Microblogs via Social Tracking.

Social Network Text Sentiment Analysis Method Based on CNN-BiGRU in Big Data Environment

Logo Information Recognition in Large-Scale Social Media Data

Word Dictionary Emoticon Dictionary SentiBank : ANP Detector Library Microblog with labeled sentiment Testing microblogs Update Update W Update g

Identifying User Interests based on Microblog Classification