Abstract:In recent years, social media has increasingly become one of the popular ways for people to consume news. As proliferation of fake news on social media has the negative impacts on individuals and society, automatic fake news detection has been explored by different research communities for combating fake news. With the development of multimedia technology, there is a phenomenon that cannot be ignored is that more and more social media news contains information with different modalities, e.g., texts, pictures and videos. The multiple information modalities show more evidence of the happening of news events and present new opportunities to detect features in fake news. First, for multimodal fake news detection task, it is a challenge of keeping the unique properties for each modality while fusing the relevant information between different modalities. Second, for some news, the information fusion between different modalities may produce the noise information which affects model's performance. Unfortunately, existing methods fail to handle these challenges. To address these problems, we propose a multimodal fake news detection framework based on Crossmodal Attention Residual and Multichannel convolutional neural Networks (CARMN). The Crossmodal Attention Residual Network (CARN) can selectively extract the relevant information related to a target modality from another source modality while maintaining the unique information of the target modality. The Multichannel Convolutional neural Network (MCN) can mitigate the influence of noise information which may be generated by crossmodal fusion component by extracting textual feature representation from original and fused textual information simultaneously. We conduct extensive experiments on four real-world datasets and demonstrate that the proposed model outperforms the state-of-the-art methods and learns more discriminable feature representations.

Cross-Modal Adversarial Contrastive Learning for Multi-Modal Rumor Detection

Multimodal Fusion Network with Contrary Latent Topic Memory for Rumor Detection

Deep visual-linguistic fusion network considering cross-modal inconsistency for rumor detection

Cross-modal Contrastive Learning for Multimodal Fake News Detection

Multimodal Adversarially Learned Inference with Factorized Discriminators

Fake News Detection Based on Cross-modal Co-attention

CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detection

BCMF: A bidirectional cross-modal fusion model for fake news detection

Multimodal Short Video Rumor Detection System Based on Contrastive Learning

A multimodal fake news detection model based on crossmodal attention residual and multichannel convolutional neural networks

Cross-modal contrastive learning for multimodal sentiment recognition

Multimodal Contrastive Learning via Uni-Modal Coding and Cross-Modal Prediction for Multimodal Sentiment Analysis

Multi-level Contrastive Learning: Hierarchical Alleviation of Heterogeneity in Multimodal Sentiment Analysis

Multi-Level Multi-Modal Cross-Attention Network for Fake News Detection

Rumor detection on social media with multimodal feature fusion

MR2: A Benchmark for Multimodal Retrieval-Augmented Rumor Detection in Social Media.

Detect Rumors in Microblog Posts for Low-Resource Domains via Adversarial Contrastive Learning

Focusing on Relevant Responses for Multi-modal Rumor Detection

Cross-Modal Search for Social Networks via Adversarial Learning

Cross-modal Consistency Learning with Fine-grained Fusion Network for Multimodal Fake News Detection

Multimodal Fusion with Recurrent Neural Networks for Rumor Detection on Microblogs