Abstract:As one of the most popular social media platforms, microblogs are ideal places for news propagation. In microblogs, tweets with both text and images are more likely to attract attention than text-only tweets. This advantage is exploited by fake news producers to publish fake news, which has a devasting impact on individuals and society. Thus, multimodal fake news detection has attracted the attention of many researchers. For news with text and image, multimodal fake news detection utilizes both text and image information to determine the authenticity of news. Most of the existing methods for multimodal fake news detection obtain a joint representation by simply concatenating a vector representation of the text and a visual representation of the image, which ignores the dependencies between them. Although there are a small number of approaches that use the attention mechanism to fuse them, they are not fine-grained enough in feature fusion. The reason is that, for a given image, there are multiple visual features and certain correlations between these features. They do not use multiple feature vectors representing different visual features to fuse with textual features, and ignore the correlations, resulting in inadequate fusion of textual features and visual features. In this paper, we propose a novel fine-grained multimodal fusion network (FMFN) to fully fuse textual features and visual features for fake news detection. Scaled dot-product attention is utilized to fuse word embeddings of words in the text and multiple feature vectors representing different features of the image, which not only considers the correlations between different visual features but also better captures the dependencies between textual features and visual features. We conduct extensive experiments on a public Weibo dataset. Our approach achieves competitive results compared with other methods for fusing visual representation and text representation, which demonstrates that the joint representation learned by the FMFN (which fuses multiple visual features and multiple textual features) is better than the joint representation obtained by fusing a visual representation and a text representation in determining fake news.

FakeSV: A Multimodal Benchmark with Rich Social Context for Fake News Detection on Short Video Platforms

VMID: A Multimodal Fusion LLM Framework for Detecting and Identifying Misinformation of Short Videos

FakingRecipe: Detecting Fake News on Short Video Platforms from the Perspective of Creative Process

Construction of Multi-Modal Social Media Dataset for Fake News Detection

Official-NV: An LLM-Generated News Video Dataset for Multimodal Fake News Detection

Multimodal Short Video Rumor Detection System Based on Contrastive Learning

FineFake: A Knowledge-Enriched Dataset for Fine-Grained Multi-Domain Fake News Detection

Multi-modal Fake News Detection on Social Media via Multi-grained Information Fusion

FMFN: Fine-Grained Multimodal Fusion Networks for Fake News Detection

Semantic‐enhanced multimodal fusion network for fake news detection

MCFEND: A Multi-source Benchmark Dataset for Chinese Fake News Detection

Vae-Clip: Unveiling Deception through Cross-Modal Models and Multi-Feature Integration in Multi-Modal Fake News Detection

BC-FND: An Approach Based on Hierarchical Bilinear Fusion and Multimodal Consistency for Fake News Detection

BCMF: A bidirectional cross-modal fusion model for fake news detection

A data-driven model for social media fake news detection

Cross-Modal Augmentation for Few-Shot Multimodal Fake News Detection

A Self-Learning Multimodal Approach for Fake News Detection

TLFND: A Multimodal Fusion Model Based on Three-Level Feature Matching Distance for Fake News Detection

FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media

Two Heads Are Better Than One: Improving Fake News Video Detection by Correlating with Neighbors

Unsupervised Domain-agnostic Fake News Detection using Multi-modal Weak Signals