Abstract:With the rapid growth of multimedia-sharing platforms (e.g. Twitter and TikTok), multimedia recommender systems have become fundamental for helping users alleviate information overload and discover items of interest. Existing multimedia recommendation methods often incorporate various auxiliary modalities (e.g., visual, textual, and acoustic) to describe item characteristics and improve task performance. However, these methods usually assume that each item is associated with complete modalities, ignoring the prevalence of missing modality issues in real-world scenarios. To deal with the challenge of missing modalities, in this paper, we propose a novel framework of Contrastive Intra- and Inter-Modality Generation (CI2MG) for enhancing incomplete multimedia recommendation. We first develop a contrastive intra- and inter-modality generation module for the missing modalities, where the intra-modality representation is updated through clustering-based hypergraph convolution and inter-modality representation is obtained by optimal transport between different modalities. To tackle the challenge of insufficient and incomplete supervision labels during intra- and inter-modality generation, a modality-aware contrastive learning paradigm is introduced based on an augmentation between the intra-modality view and inter-modality view. Furthermore, to learn task-related representations from the generative modalities and further improve the performance of recommendation, we design an enhanced multimedia recommendation module to alleviate the influences driven by task-irrelevant noise. Extensive experiments on real-world datasets show the superiority of our proposed CI2MG framework in offering great potential for personalized multimedia recommendation over the state-of-the-art baselines regarding Recall, NDCG, and Precision metrics.

Enhancing Adversarial Robustness of Multi-modal Recommendation via Modality Balancing

Contrastive Intra- and Inter-Modality Generation for Enhancing Incomplete Multimedia Recommendation

Improving Item-side Fairness of Multimodal Recommendation Via Modality Debiasing

Adversarial Training Towards Robust Multimedia Recommender System

Asymmetric Reinforcing against Multi-modal Representation Bias

Mutual-modality Adversarial Attack with Semantic Perturbation

Prompt-based and Weak-Modality Enhanced Multimodal Recommendation

Multi-modal visual adversarial Bayesian personalized ranking model for recommendation

Quantifying and Enhancing Multi-modal Robustness with Modality Preference

Robust multimedia recommender system based on dynamic collaborative filtering and directed adversarial learning

Robust-MSA: Understanding the Impact of Modality Noise on Multimodal Sentiment Analysis

Multi-Modal Self-Supervised Learning for Recommendation

On Adversarial Robustness of Large-scale Audio Visual Learning

Enhance Modality Robustness in Text-Centric Multimodal Alignment with Adversarial Prompting

FMMRec: Fairness-aware Multimodal Recommendation

Towards Robust Recommendation: A Review and an Adversarial Robustness Evaluation Library

Understanding and Measuring Robustness of Multimodal Learning

Modality-Guided Collaborative Filtering for Recommendation

Robust multi-objective visual bayesian personalized ranking for multimedia recommendation

Multiloss Adversarial Attacks for Multimodal Remote Sensing Image Classification

Natural Language Explainable Recommendation with Robustness Enhancement