Abstract:More and more social media images are produced on the Internet. Social image classification is an important task in many fields, such as public opinion analysis and sensitive image detection. Although convolutional neural networks (CNN) have achieved amazing achievements in the field of image classification. However, social images are special multi-label images. social image classification still has a lot of space for improvement. In one aspect, we find that many methods using CNN networks for classification tasks mainly use the top-level features, which contain limited information. In the real world, social images usually contain multiple semantic objects with different scales. Using the top-level features to classify different objects may not be the most suitable. Therefore, we propose a multi-branch prediction network. Since the shallow structure of the CNN network can capture some low-level features of the image, such as boundary, color, and so on, while the deeper features can capture some high-level semantics of the image, so different branches can get different representations for multi-label image classification. Subsequently, we also use the feature fusion module for each branch feature, using other branch features as a slight supplement. An efficient attention mechanism is introduced after the fusion module of each branch, which embeds the position information of the feature map into the channel attention so that the branch prediction can locate and identify the object area more accurately. In particular, a multi-branch independent prediction strategy is adopted in this work. The branch result with the highest prediction probability is regarded as the prediction of the whole network. Experiments show that the accuracy of the classification method proposed in this paper is better than other methods for multi-label classification problems.

Classify Social Image by Integrating Multi-Modal Content

Multi-modal Learning for Social Image Classification

Image Classification Method by Combining Multi-features and Sparse Coding

Multi-modal Kernel Ridge Regression for Social Image Classification

Social Image Sentiment Analysis by Exploiting Multimodal Content and Heterogeneous Relations

Social Image-text Sentiment Classification With Cross-Modal Consistency and Knowledge Distillation

Multimodal Classification for Analysing Social Media

Fusing Multi-type Features with Distance Metric Learning for Social Image Application

Efficient Multi-Modal Hypergraph Learning for Social Image Classification with Complex Label Correlations

Integrating Multiple Types of Features for Event Identification in Social Images

Heterogeneous Image Features Integration Via Multi-modal Semi-supervised Learning Model

Multimodal Learning for Multi-Label Image Classification.

Integration of Image Feature and Word Relevance: Toward Automatic Image Annotation in Cyber-Physical-Social Systems

Transfer Correlation Between Textual Content to Images for Sentiment Analysis

Multi-modal microblog classification via multi-task learning

Multimodal Learning of Social Image Representation by Exploiting Social Relations

Multi-branch Prediction Network for Multi-label Social Image Classification.

Sentiment Analysis of Social Images Via Hierarchical Deep Fusion of Content and Links.

Improving Fine-grained Image Classification with Multimodal Information

Cross-modal image sentiment analysis via deep correlation of textual semantic

From Content to Links: Social Image Embedding with Deep Multimodal Model.