A Reinforcement Learning Approach for Personalized Diversity in Feeds Recommendation.

Li He,Kangqi Luo,Zhuoye Ding,Hang Shao,Bing Bai
DOI: https://doi.org/10.1007/978-981-99-9119-8_42
2024-01-01
Abstract:Feeds recommendation has been widely used in various applications, such as e-commerce site, where users can constantly browse products generated by never-ending feeds. It’s important to not only consider instant metrics but also pay more attention to long-term user engagement. In this paper, we focus on optimizing user browsing depth, which represents users’ willingness to stay within the e-commerce feed streams. By analyzing the ranking and re-ranking stages, we find that the re-ranking stage is a suitable phase for maximizing user browsing depth. First, we evaluate the current status of our used re-ranking module and identify that the fixed diversity rule neglects unique propensity to the degree of diversity in each user request. Hence there is a need to personalize diversity in the granularity of user requests. Then, we note that the personalized diversity process of user request granularity can be modelled as a Markov decision process (MDP). Finally, by solving three issues of MDP elements design, acquisition of interaction data, off-policy learning and policy selection, we propose a Personalized Diversity Re-ranking Model in the granularity of user request (PDRM-request) based on reinforcement learning. We conduct offline experiments and deploy the PDRM-request model in a live e-commerce site to perform A/B testing. The results show that the our approach achieves deeper user browsing depth and more diversified recommended lists than the existing baseline.
What problem does this paper attempt to address?