Abstract:Social media misinformation harms individuals and societies and is potentialized by fast-growing multi-modal content (i.e., texts and images), which accounts for higher "credibility" than text-only news pieces. Although existing supervised misinformation detection methods have obtained acceptable performances in key setups, they may require large amounts of labeled data from various events, which can be time-consuming and tedious. In turn, directly training a model by leveraging a publicly available dataset may fail to generalize due to domain shifts between the training data (a.k.a. source domains) and the data from target domains. Most prior work on domain shift focuses on a single modality (e.g., text modality) and ignores the scenario where sufficient unlabeled target domain data may not be readily available in an early stage. The lack of data often happens due to the dynamic propagation trend (i.e., the number of posts related to fake news increases slowly before catching the public attention). We propose a novel robust domain and cross-modal approach (\textbf{RDCM}) for multi-modal misinformation detection. It reduces the domain shift by aligning the joint distribution of textual and visual modalities through an inter-domain alignment module and bridges the semantic gap between both modalities through a cross-modality alignment module. We also propose a framework that simultaneously considers application scenarios of domain generalization (in which the target domain data is unavailable) and domain adaptation (in which unlabeled target domain data is available). Evaluation results on two public multi-modal misinformation detection datasets (Pheme and Twitter Datasets) evince the superiority of the proposed model. The formal implementation of this paper can be found in this link: <a class="link-external link-https" href="https://github.com/less-and-less-bugs/RDCM" rel="external noopener nofollow">this https URL</a>

Robust Domain Misinformation Detection via Multi-modal Feature Alignment

Cross-Domain Rumor Detection Based on Dual-Modal Domain Alignment

Towards Cross-Lingual Multi-Modal Misinformation Detection for E-Commerce Management

Detecting Misinformation in Multimedia Content through Cross-Modal Entity Consistency: A Dual Learning Approach

MDMN: Multi-task and Domain Adaptation Based Multi-modal Network for Early Rumor Detection

MRDCA: a multimodal approach for fine-grained fake news detection through integration of RoBERTa and DenseNet based upon fusion mechanism of co-attention

LIMFA: Label-Irrelevant Multi-Domain Feature Alignment-Based Fake News Detection for Unseen Domain

Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMs

MMIDR: Teaching Large Language Model to Interpret Multimodal Misinformation via Knowledge Distillation

RAEmoLLM: Retrieval Augmented LLMs for Cross-Domain Misinformation Detection Using In-Context Learning based on Emotional Information

Unsupervised Cross-Domain Rumor Detection from Multiple Sources Based on RoBERTa and Multi-CNN.

Multi-modal Rumour Detection Using Bilinear Pooling and Domain Adversarial Neural Networks.

Collaborative Mixture-of-Experts Model for Multi-Domain Fake News Detection

Unsupervised Domain Adaptation for COVID-19 Information Service with Contrastive Adversarial Domain Mixup

A Semi-Supervised Framework for Misinformation Detection

Cross-domain Multi-modal Few-shot Object Detection via Rich Text

Inconsistent Matters: A Knowledge-guided Dual-consistency Network for Multi-modal Rumor Detection

RoMo: Robust Unsupervised Multimodal Learning with Noisy Pseudo Labels

Harnessing the Power of Text-image Contrastive Models for Automatic Detection of Online Misinformation

Unsupervised Domain Adaptation Via Contrastive Adversarial Domain Mixup: A Case Study on COVID-19

RaCMC: Residual-Aware Compensation Network with Multi-Granularity Constraints for Fake News Detection