Abstract:More and more social media images are produced on the Internet. Social image classification is an important task in many fields, such as public opinion analysis and sensitive image detection. Although convolutional neural networks (CNN) have achieved amazing achievements in the field of image classification. However, social images are special multi-label images. social image classification still has a lot of space for improvement. In one aspect, we find that many methods using CNN networks for classification tasks mainly use the top-level features, which contain limited information. In the real world, social images usually contain multiple semantic objects with different scales. Using the top-level features to classify different objects may not be the most suitable. Therefore, we propose a multi-branch prediction network. Since the shallow structure of the CNN network can capture some low-level features of the image, such as boundary, color, and so on, while the deeper features can capture some high-level semantics of the image, so different branches can get different representations for multi-label image classification. Subsequently, we also use the feature fusion module for each branch feature, using other branch features as a slight supplement. An efficient attention mechanism is introduced after the fusion module of each branch, which embeds the position information of the feature map into the channel attention so that the branch prediction can locate and identify the object area more accurately. In particular, a multi-branch independent prediction strategy is adopted in this work. The branch result with the highest prediction probability is regarded as the prediction of the whole network. Experiments show that the accuracy of the classification method proposed in this paper is better than other methods for multi-label classification problems.

Multi-modal Learning for Social Image Classification

Classify Social Image by Integrating Multi-Modal Content

Multi-modal Kernel Ridge Regression for Social Image Classification

Efficient Multi-Modal Hypergraph Learning for Social Image Classification with Complex Label Correlations

Social Image-text Sentiment Classification With Cross-Modal Consistency and Knowledge Distillation

Fusing Multi-type Features with Distance Metric Learning for Social Image Application

Social Image Sentiment Analysis by Exploiting Multimodal Content and Heterogeneous Relations

Multimodal Classification for Analysing Social Media

Multimodal Learning for Multi-Label Image Classification.

Cross-Modal Image-Tag Relevance Learning for Social Images

Multimodal Learning of Social Image Representation by Exploiting Social Relations

Multi-branch Prediction Network for Multi-label Social Image Classification.

Multi-modal microblog classification via multi-task learning

Multimodal Learning for View-Based 3D Object Classification

Fusion of Multi-modal Information of User Profile Across Social Networks for User Identification

Heterogeneous Image Features Integration Via Multi-modal Semi-supervised Learning Model

Learning Deep Feature Fusion for Group Images Classification.

Multi-model Fusion Metric Learning for Image Set Classification

A multimodal sentiment recognition method based on attention mechanism

Improving Fine-grained Image Classification with Multimodal Information

TCGM: an Information-Theoretic Framework for Semi-Supervised Multi-Modality Learning