Dealing with Missing Modalities in Multimodal Recommendation: a Feature Propagation-based Approach

Daniele Malitesta,Emanuele Rossi,Claudio Pomo,Fragkiskos D. Malliaros,Tommaso Di Noia
2024-03-29
Abstract:Multimodal recommender systems work by augmenting the representation of the products in the catalogue through multimodal features extracted from images, textual descriptions, or audio tracks characterising such products. Nevertheless, in real-world applications, only a limited percentage of products come with multimodal content to extract meaningful features from, making it hard to provide accurate recommendations. To the best of our knowledge, very few attention has been put into the problem of missing modalities in multimodal recommendation so far. To this end, our paper comes as a preliminary attempt to formalise and address such an issue. Inspired by the recent advances in graph representation learning, we propose to re-sketch the missing modalities problem as a problem of missing graph node features to apply the state-of-the-art feature propagation algorithm eventually. Technically, we first project the user-item graph into an item-item one based on co-interactions. Then, leveraging the multimodal similarities among co-interacted items, we apply a modified version of the feature propagation technique to impute the missing multimodal features. Adopted as a pre-processing stage for two recent multimodal recommender systems, our simple approach performs better than other shallower solutions on three popular datasets.
Information Retrieval
What problem does this paper attempt to address?
### Problems the Paper Attempts to Solve This paper aims to address the issue of missing modalities in multimodal recommendation systems. Specifically: 1. **Problem Definition**: The paper first attempts to formally define the problem of missing modalities in multimodal recommendation systems and proposes a method to simulate this setting by randomly selecting a portion of items from popular recommendation datasets as items with missing modalities. 2. **Method Proposal**: Inspired by the latest advances in graph representation learning, the authors reshape the problem of missing modalities into a problem of missing graph node features and apply a state-of-the-art feature propagation algorithm (FeatProp) to fill in the missing multimodal features. This method can serve as a preprocessing stage for any multimodal recommendation system, being simple and non-intrusive to the training process. 3. **Experimental Validation**: Through extensive experiments on three popular Amazon catalog datasets (a total of 1080 experiments), the paper validates the effectiveness of the FeatProp algorithm. The experiments include two state-of-the-art multimodal recommendation systems (MMSSL and FREEDOM), four methods for filling in missing modalities (Zeros, Mean, Random, and FeatProp), and nine different proportions of missing modalities. In summary, the paper mainly focuses on how to effectively handle the situation where some items in multimodal recommendation systems lack multimodal features and proposes a new feature propagation-based method to solve this problem.