Abstract:With the development of the Internet, users can freely publish posts on various social media platforms, which offers great convenience for keeping abreast of the world. However, posts usually carry many rumors, which require plenty of manpower for monitoring. Owing to the success of modern machine learning techniques, especially deep learning models, we tried to detect rumors as a classification problem automatically. Early attempts have always focused on building classifiers relying on image or text information, i.e., single modality in posts. Thereafter, several multimodal detection approaches employ an early or late fusion operator for aggregating multiple source information. Nevertheless, they only take advantage of multimodal embeddings for fusion and ignore another important detection factor, i.e., the intermodal inconsistency between modalities. To solve this problem, we develop a novel deep visual-linguistic fusion network (DVLFN) considering cross-modal inconsistency, which detects rumors by comprehensively considering modal aggregation and contrast information. Specifically, the DVLFN first utilizes visual and textual deep encoders, i.e., Faster R-CNN and bidirectional encoder representations from transformers, to extract global and regional embeddings for image and text modalities. Then, it predicts posts' authenticity from two aspects: (1) intermodal inconsistency, which employs the Wasserstein distance to efficiently measure the similarity between regional embeddings of different modalities, and (2) modal aggregation, which experimentally employs the early fusion to aggregate two modal embeddings for prediction. Consequently, the DVLFN can compose the final prediction based on the modal fusion and inconsistency measure. Experiments are conducted on three real-world multimedia rumor detection datasets collected from Reddit, GoodNews, and Weibo. The results validate the superior performance of the proposed DVLFN.

Distributed Rumor Source Detection Via Boosted Federated Learning

Interpretable Graph Neural Network for Social Media Rumor Detection

Federated Graph Attention Network for Rumor Detection

Region-enhanced Deep Graph Convolutional Networks for Rumor Detection

Multiple Rumor Source Detection with Graph Convolutional Networks.

DLRD: Dual-Level Network for Rumor Detection on Geo-Textual Data

Catch Me if You Can: A Participant-Level Rumor Detection Framework Via Fine-Grained User Representation Learning.

Rumor Detection with a novel graph neural network approach

Rumor Localization, Detection and Prediction in Social Network

Rumor Detection on Social Media with Bi-Directional Graph Convolutional Networks

RP-DNN: A Tweet level propagation context based deep neural networks for early rumor detection in Social Media

Rumor detection on social media using hierarchically aggregated feature via graph neural networks

Graph Convolution-Based Joint Learning of Rumor with Content, User Credibility, Propagation Context, and Cognitive as Well as Emotion Signals

Deep visual-linguistic fusion network considering cross-modal inconsistency for rumor detection

A Bi-GRU-DSA-based social network rumor detection approach

Towards Trustworthy Rumor Detection with Interpretable Graph Structural Learning

Cross-Domain Social Rumor-Propagation Model Based on Transfer Learning

Modeling microscopic and macroscopic information diffusion for rumor detection

Rumor Detection with Diverse Counterfactual Evidence

Modeling the Uncertainty of Information Propagation for Rumor Detection: A Neuro-Fuzzy Approach

Research Status of Deep Learning Methods for Rumor Detection