Learning Without Missing-At-Random Prior Propensity-A Generative Approach for Recommender Systems
Yuanbo Xu,Fuzhen Zhuang,En Wang,Chaozhuo Li,Jie Wu
DOI: https://doi.org/10.1109/tkde.2024.3490593
IF: 9.235
2024-01-01
IEEE Transactions on Knowledge and Data Engineering
Abstract:A common assumption in the literature is that the missing ratings are missing at random (MAR), meaning that the probability of observing a rating does not depend on its value. However, this assumption is often violated in real-world scenarios, where users tend to provide ratings for items they like or dislike more than average, leading to a missing not at random (MNAR) situation. To address this problem, some researchers have proposed to use explicit MAR feedbacks to estimate the propensities of unobserved implicit MNAR feedbacks. However, collecting explicit MAR feedbacks is costly and time-consuming and may not reflect users' true preferences. Moreover, most of these methods have only been tested on synthetic or small-scale datasets, and their applicability and effectiveness in real-world settings without MAR feedbacks remain unclear. To this end, we aim to predict MNAR ratings without MAR prior propensities by exploring the consistency between MAR and MNAR feedbacks and bridging the gap between them. From the empirical study and preliminary experiment, we hypothesize that user preferences can be treated as the common prior propensity for both MAR and MNAR generative processes. In this way, we extend this hypothesis to a more general MNAR scenario: user preferences learned from MNAR can partially substitute for the prior propensities derived from MAR feedbacks for MNAR recommendation tasks. To validate our hypothesis and approach, we develop a lightweight iterative probabilistic matrix factorization framework (lightIPMF) as a practical method of our methodology, utilizing user preferences extracted from MNAR, not MAR, to estimate MNAR feedbacks. Finally, the experimental results show that modeling user preferences can effectively improve MNAR feedback estimation without MAR feedback, and our proposed lightIPMF outperforms the state-of-the-art MNAR methods in predicting MNAR feedbacks.