Deep Latent Emotion Network for Multi-Task Learning

Huangbin Zhang,Chong Zhao,Yu Zhang,Danlei Wang,Haichao Yang
DOI: https://doi.org/10.48550/arXiv.2104.08716
2021-04-18
Abstract:Feed recommendation models are widely adopted by numerous feed platforms to encourage users to explore the contents they are interested in. However, most of the current research simply focus on targeting user's preference and lack in-depth study of avoiding objectionable contents to be frequently recommended, which is a common reason that let user detest. To address this issue, we propose a Deep Latent Emotion Network (DLEN) model to extract latent probability of a user preferring a feed by modeling multiple targets with semi-supervised learning. With this method, the conflicts of different targets are successfully reduced in the training phase, which improves the training accuracy of each target effectively. Besides, by adding this latent state of user emotion to multi-target fusion, the model is capable of decreasing the probability to recommend objectionable contents to improve user retention and stay time during online testing phase. DLEN is deployed on a real-world multi-task feed recommendation scenario of Tencent QQ-Small-World with a dataset containing over a billion samples, and it exhibits a significant performance advantage over the SOTA MTL model in offline evaluation, together with a considerable increase by 3.02% in view-count and 2.63% in user stay-time in production. Complementary offline experiments of DLEN model on a public dataset also repeat improvements in various scenarios. At present, DLEN model has been successfully deployed in Tencent's feed recommendation system.
Artificial Intelligence,Information Retrieval
What problem does this paper attempt to address?
The main problems that this paper attempts to solve are the problems existing in the multi - task learning (MTL) models in current recommendation systems when dealing with information - flow recommendation, specifically including: 1. **Avoiding frequently recommending offensive content**: Most of the existing research mainly focuses on optimizing according to users' preferences, while lacking in - depth research on avoiding frequently recommending content that users dislike. This may lead to the loss of users because they often see content they don't like. 2. **Conflicts between different tasks**: In multi - task learning, due to the large differences in the goals of different tasks, conflicts are likely to occur during the learning process of shared parameters, which affects the overall performance of the model. 3. **Complexity of negative samples**: In the information - flow recommendation system, negative samples (i.e., content with which users have not interacted) account for the vast majority, and these negative samples may contain different user emotions (such as dislike, indifference, acceptance or even like). Traditional methods are difficult to distinguish these different types of negative samples, resulting in model training bias. To solve the above problems, the paper proposes a multi - task learning model based on the Deep Latent Emotion Network (DLEN). DLEN models users' latent emotional tendencies by introducing the Bayesian probability formula, thereby reducing the conflicts between different tasks and improving the utilization efficiency of negative samples of the model, and finally increasing the user retention rate and dwell time. ### Specific solutions - **Introducing latent emotional states**: DLEN takes users' latent emotional tendencies as hidden targets and couples them with other explicit behavior targets for training through the Bayesian probability formula. - **Reducing task conflicts**: Compared with existing multi - task models (such as MMOE and PLE), DLEN reduces the conflicts between different tasks by sharing targets and improves the training effect. - **Making full use of negative samples**: DLEN can distinguish different types of negative samples, avoid simply regarding all negative samples as content that users dislike, and thus more accurately capture users' true preferences. ### Experimental verification The paper verifies the effectiveness of the DLEN model through online and offline experiments. The experimental results show that DLEN is significantly superior to existing multi - task learning models in multiple evaluation indicators, especially performing excellently in key indicators such as click - through rate, like rate, dwell time and page views. In conclusion, this paper aims to improve the multi - task learning model by introducing users' latent emotional states, so as to better handle the complex problems in information - flow recommendation and enhance user experience and business value.