Abstract:As one of the most popular social media platforms, microblogs are ideal places for news propagation. In microblogs, tweets with both text and images are more likely to attract attention than text-only tweets. This advantage is exploited by fake news producers to publish fake news, which has a devasting impact on individuals and society. Thus, multimodal fake news detection has attracted the attention of many researchers. For news with text and image, multimodal fake news detection utilizes both text and image information to determine the authenticity of news. Most of the existing methods for multimodal fake news detection obtain a joint representation by simply concatenating a vector representation of the text and a visual representation of the image, which ignores the dependencies between them. Although there are a small number of approaches that use the attention mechanism to fuse them, they are not fine-grained enough in feature fusion. The reason is that, for a given image, there are multiple visual features and certain correlations between these features. They do not use multiple feature vectors representing different visual features to fuse with textual features, and ignore the correlations, resulting in inadequate fusion of textual features and visual features. In this paper, we propose a novel fine-grained multimodal fusion network (FMFN) to fully fuse textual features and visual features for fake news detection. Scaled dot-product attention is utilized to fuse word embeddings of words in the text and multiple feature vectors representing different features of the image, which not only considers the correlations between different visual features but also better captures the dependencies between textual features and visual features. We conduct extensive experiments on a public Weibo dataset. Our approach achieves competitive results compared with other methods for fusing visual representation and text representation, which demonstrates that the joint representation learned by the FMFN (which fuses multiple visual features and multiple textual features) is better than the joint representation obtained by fusing a visual representation and a text representation in determining fake news.

Novel Visual and Statistical Image Features for Microblogs News Verification

News Verification by Exploiting Conflicting Social Viewpoints in Microblogs.

Sentiment Analysis on Microblogging by Integrating Text and Image Features

FMFN: Fine-Grained Multimodal Fusion Networks for Fake News Detection

Exploring the Role of Visual Content in Fake News Detection

Predicting image credibility in fake news over social media using multi-modal approach

Exploiting Multi-domain Visual Information for Fake News Detection

News Feature Extraction for Events on Social Network Platforms

Feature Importance Analysis for Spammer Detection in Sina Weibo

Mining Significant Microblogs for Misinformation Identification: An Attention-based Approach

Detecting Event Rumors on Sina Weibo Automatically.

News Credibility Evaluation on Microblog with a Hierarchical Propagation Model.

Sentiment Analysis for Images on Microblogging by Integrating Textual Information with Multiple Kernel Learning.

Including Images into Message Veracity Assessment in Social Media

Do Photos Help Express Our Feelings: Incorporating Multimodal Features into Microblog Sentiment Analysis.

Fake news detection for epidemic emergencies via deep correlations between text and images

False News Detection on Social Media

Multi-level word features based on CNN for fake news detection in cultural communication

Detecting Spam in Chinese Microblogs - A Study on Sina Weibo

A Joint Fake News Detection Model based on Multi-Features

A Unified Approach of Detecting Misleading Images via Tracing its Instances on Web and Analysing its Past Context for the Verification of Content