Abstract:Recommender System (RS) is an important online application that affects billions of users every day. The mainstream RS ranking framework is composed of two parts: a Multi-Task Learning model (MTL) that predicts various user feedback, i.e., clicks, likes, sharings, and a Multi-Task Fusion model (MTF) that combines the multi-task outputs into one final ranking score with respect to user satisfaction. There has not been much research on the fusion model while it has great impact on the final recommendation as the last crucial process of the ranking. To optimize long-term user satisfaction rather than obtain instant returns greedily, we formulate MTF task as Markov Decision Process (MDP) within a recommendation session and propose a Batch Reinforcement Learning (RL) based Multi-Task Fusion framework (BatchRL-MTF) that includes a Batch RL framework and an online exploration. The former exploits Batch RL to learn an optimal recommendation policy from the fixed batch data offline for long-term user satisfaction, while the latter explores potential high-value actions online to break through the local optimal dilemma. With a comprehensive investigation on user behaviors, we model the user satisfaction reward with subtle heuristics from two aspects of user stickiness and user activeness. Finally, we conduct extensive experiments on a billion-sample level real-world dataset to show the effectiveness of our model. We propose a conservative offline policy estimator (Conservative-OPEstimator) to test our model offline. Furthermore, we take online experiments in a real recommendation environment to compare performance of different models. As one of few Batch RL researches applied in MTF task successfully, our model has also been deployed on a large-scale industrial short video platform, serving hundreds of millions of users.

Single-shot Feature Selection for Multi-task Recommendations

MultiFS: Automated Multi-Scenario Feature Selection in Deep Recommender Systems.

Towards Personalized Federated Multi-Scenario Multi-Task Recommendation

Multi-task Feature Learning for Social Recommendation

Personalized Multi-task Training for Recommender System

MvFS: Multi-view Feature Selection for Recommender System

Multi-Scenario and Multi-Task Aware Feature Interaction for Recommendation System

Multi-Task Deep Recommender Systems: A Survey

Feature Selection for Multimedia Analysis by Sharing Information among Multiple Tasks

Multi-modal Recommendation Based on Knowledge Graph

Efficient Multi-task Prompt Tuning for Recommendation

Multi-Task Feature Learning for Knowledge Graph Enhanced Recommendation

An Empirical Study of Training ID-Agnostic Multi-modal Sequential Recommenders

Multi-Faceted Hierarchical Multi-Task Learning for a Large Number of Tasks with Multi-dimensional Relations

Learning Recommender Systems from Multi-Behavior Data

Neural Multi-task Recommendation from Multi-behavior Data.

Multi-Task Recommendations with Reinforcement Learning

Multi-Task Fusion via Reinforcement Learning for Long-Term User Satisfaction in Recommender Systems

Divide and Conquer: Towards Better Embedding-based Retrieval for Recommender Systems From a Multi-task Perspective

DTN: Deep Multiple Task-specific Feature Interactions Network for Multi-Task Recommendation

Online Weighted Multi-Task Feature Selection