Abstract:Recommender system usually faces popularity bias. From the popularity distribution shift perspective, the normal paradigm trained on exposed items (most are hot items) identifies that recommending popular items more frequently can achieve lower loss, thus injecting popularity information into item property embedding, e.g., id embedding. From the long-tail distribution shift perspective, the sparse interactions of long-tail items lead to insufficient learning of them. The resultant distribution discrepancy between hot and long-tail items would not only inherit the bias, but also amplify the bias. Existing work addresses this issue with inverse propensity scoring (IPS) or causal embeddings. However, we argue that not all popularity biases mean bad effects, i.e., some items show higher popularity due to better quality or conform to current trends, which deserve more recommendations. Blindly seeking unbiased learning may inhibit high-quality or fashionable items. To make better use of the popularity bias, we propose a co-training disentangled domain adaptation network (CD$^2$AN), which can co-train both biased and unbiased models. Specifically, for popularity distribution shift, CD$^2$AN disentangles item property representation and popularity representation from item property embedding. For long-tail distribution shift, we introduce additional unexposed items (most are long-tail items) to align the distribution of hot and long-tail item property representations. Further, from the instances perspective, we carefully design the item similarity regularization to learn comprehensive item representation, which encourages item pairs with more effective co-occurrences patterns to have more similar item property representations. Based on offline evaluations and online A/B tests, we show that CD$^2$AN outperforms the existing debiased solutions. Currently, CD$^2$AN has been successfully deployed at Mobile Taobao App and handling major online traffic.

UTSP: User-Based Two-Step Recommendation with Popularity Normalization Towards Diversity and Novelty

Co-training Disentangled Domain Adaptation Network for Leveraging Popularity Bias in Recommenders

A Personalized Recommendation Algorithm Via Biased Random Walk

Interest Before Liking: Two-step Recommendation Approaches

Novelty and Diversity in Top-N Recommendation - Analysis and Evaluation

User Popularity Preference Aware Sequential Recommendation

Relieving Popularity Bias in Interactive Recommendation: A Diversity-Novelty-Aware Reinforcement Learning Approach

A Meta-Level Hybrid Recommendation Method Based on User Novelty.

A Joint Optimization Approach for Personalized Recommendation Diversification.

A Time-aware Hybrid Algorithm for Online Recommendation Services

Avoiding monotony: improving the diversity of recommendation lists

Taxonomy Based Personalized News Recommendation: Novelty And Diversity

Enhancing Recommendation Diversity and Novelty with Bi-LSTM and Mean Shift Clustering

Serendipitous Personalized Ranking For Top-N Recommendation

Evaluating the Diversity of Top-N Recommendations

Dynamic Personalized Recommendation on Sparse Data

Personalized Re-ranking for Improving Diversity in Live Recommender Systems

A joint optimization approach for personalized recommendation diversification

Neural Serendipity Recommendation: Exploring the Balance between Accuracy and Novelty with Sparse Explicit Feedback

Relevance meets Diversity: A User-Centric Framework for Knowledge Exploration through Recommendations

Choice-Based Recommender Systems: A Unified Approach to Achieving Relevancy and Diversity