Abstract:Recommender system usually faces popularity bias. From the popularity distribution shift perspective, the normal paradigm trained on exposed items (most are hot items) identifies that recommending popular items more frequently can achieve lower loss, thus injecting popularity information into item property embedding, e.g., id embedding. From the long-tail distribution shift perspective, the sparse interactions of long-tail items lead to insufficient learning of them. The resultant distribution discrepancy between hot and long-tail items would not only inherit the bias, but also amplify the bias. Existing work addresses this issue with inverse propensity scoring (IPS) or causal embeddings. However, we argue that not all popularity biases mean bad effects, i.e., some items show higher popularity due to better quality or conform to current trends, which deserve more recommendations. Blindly seeking unbiased learning may inhibit high-quality or fashionable items. To make better use of the popularity bias, we propose a co-training disentangled domain adaptation network (CD$^2$AN), which can co-train both biased and unbiased models. Specifically, for popularity distribution shift, CD$^2$AN disentangles item property representation and popularity representation from item property embedding. For long-tail distribution shift, we introduce additional unexposed items (most are long-tail items) to align the distribution of hot and long-tail item property representations. Further, from the instances perspective, we carefully design the item similarity regularization to learn comprehensive item representation, which encourages item pairs with more effective co-occurrences patterns to have more similar item property representations. Based on offline evaluations and online A/B tests, we show that CD$^2$AN outperforms the existing debiased solutions. Currently, CD$^2$AN has been successfully deployed at Mobile Taobao App and handling major online traffic.

Relieving Popularity Bias in Interactive Recommendation: A Diversity-Novelty-Aware Reinforcement Learning Approach

Covering Diversification and Fairness for Better Recommendation (Short Paper).

Diversity-Promoting Deep Reinforcement Learning for Interactive Recommendation

Diversity-Aware Top-N Recommendation: A Deep Reinforcement Learning Way

Co-training Disentangled Domain Adaptation Network for Leveraging Popularity Bias in Recommenders

A Reinforcement Learning Approach for Personalized Diversity in Feeds Recommendation.

User Interest Dynamics on Personalized Recommendation

Deep Reinforcement Learning for List-wise Recommendations

Diversify or Not: Dynamic Diversification for Personalized Recommendation

Personalized Re-ranking for Improving Diversity in Live Recommender Systems

Personalized Diversification for Neural Re-ranking in Recommendation.

Hierarchical Reinforcement Learning for Modeling User Novelty-Seeking Intent in Recommender Systems

DLCRec: A Novel Approach for Managing Diversity in LLM-Based Recommender Systems

Sim-to-Real Interactive Recommendation via Off-Dynamics Reinforcement Learning

Diversity-aware Deep Ranking Network for Recommendation

A Novel Framework for Improving Recommender Diversity.

Balancing Accuracy and Fairness for Interactive Recommendation with Reinforcement Learning

Towards Long-term Fairness in Recommendation

Improving ranking function and diversification in interactive recommendation systems based on deep reinforcement learning

Distribution-Aware Diversification for Personalized Re-ranking in Recommendation

Alleviating Matthew Effect of Offline Reinforcement Learning in Interactive Recommendation