Abstract:Reinforcement learning methods have been used to optimize long-term user engagement in recommendation systems. However, existing reinforcement learning-based recommendation systems do not fully exploit the relevance of individual user behavior across different platforms. One potential solution is to aggregate data from various platforms in a centralized location and use the aggregated data for training. However, this approach raises economic and legal concerns, including increased communication costs and potential threats to user privacy. To address these challenges, we propose \textbf{FedSlate}, a federated reinforcement learning recommendation algorithm that effectively utilizes information that is prohibited from being shared at a legal level. We employ the SlateQ algorithm to assist FedSlate in learning users' long-term behavior and evaluating the value of recommended content. We extend the existing application scope of recommendation systems from single-user single-platform to single-user multi-platform and address cross-platform learning challenges by introducing federated learning. We use RecSim to construct a simulation environment for evaluating FedSlate and compare its performance with state-of-the-art benchmark recommendation models. Experimental results demonstrate the superior effects of FedSlate over baseline methods in various environmental settings, and FedSlate facilitates the learning of recommendation strategies in scenarios where baseline methods are completely inapplicable. Code is available at \textit{<a class="link-external link-https" href="https://github.com/TianYaDY/FedSlate" rel="external noopener nofollow">this https URL</a>}.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is that the existing recommendation systems based on reinforcement learning fail to fully utilize the correlation of individual behaviors of users among different platforms. Specifically: 1. **Cross - platform data sharing and privacy protection**: Existing recommendation systems usually operate independently on one platform and cannot effectively utilize users' behavioral data on multiple platforms. Although directly integrating data from different platforms can improve the recommendation effect, it will cause economic and legal problems, such as increasing communication costs and potential threats to users' privacy. Therefore, how to effectively utilize cross - platform user behavior data without violating users' privacy is an urgent problem to be solved. 2. **Scalability of single - platform and multi - platform recommendation systems**: Traditional reinforcement learning recommendation algorithms are mainly designed for a single platform and are difficult to be directly extended to multi - platform scenarios. For example, although the SlateQ algorithm can effectively handle recommendation problems on a single platform, in a multi - platform environment, its performance will be limited due to data isolation among platforms. How to design a recommendation algorithm that can adapt to a multi - platform environment is another key challenge. To solve the above problems, the author proposes FedSlate, a reinforcement learning recommendation algorithm based on federated learning. FedSlate solves these problems in the following ways: - **Utilization of cross - platform data under the federated learning framework**: FedSlate adopts federated learning technology to effectively utilize cross - platform user behavior data without directly sharing user data. The agents on each platform calculate the local Q - values and send these values to the central server, and the central server then calculates the global Q - values according to these local Q - values for the selection of content recommendation. - **Learning of multi - platform recommendation strategies**: By introducing the "Q - value sharing" mechanism, FedSlate can not only track users' long - term behaviors but also evaluate the impact of recommended content on different platforms on users' activities. This enables FedSlate to optimize recommendation strategies in a multi - platform environment, and even if some platforms cannot directly obtain user feedback, they can also benefit from federated learning. - **Reduction of communication overhead and protection of privacy**: The design of FedSlate takes into account communication efficiency and user privacy protection. By only transmitting necessary Q - values instead of original user data, the communication overhead is reduced and the security of users' privacy is ensured. In summary, FedSlate aims to solve multiple challenges such as cross - platform data utilization, recommendation strategy optimization, and communication overhead and privacy protection faced by recommendation systems in a multi - platform environment through federated learning technology.

FedSlate:A Federated Deep Reinforcement Learning Recommender System

FairRec: Fairness Testing for Deep Recommender Systems

Slate-Aware Ranking for Recommendation

FedRL: A Reinforcement Learning Federated Recommender System for Efficient Communication Using Reinforcement Selector and Hypernet Generator

Efficient and Robust Regularized Federated Recommendation

Diffusion Model for Slate Recommendation

FedGR: Cross-platform federated group recommendation system with hypergraph neural networks

FedRec: Federated Recommendation With Explicit Feedback

AeroRec: an Efficient On-Device Recommendation Framework Using Federated Self-Supervised Knowledge Distillation

Federated recommender systems based on deep learning: The experimental comparisons of deep learning algorithms and federated learning aggregation strategies

LLM-based Federated Recommendation

Personalized Federated Recommendation via Joint Representation Learning, User Clustering, and Model Adaptation

Beyond Similarity: Personalized Federated Recommendation with Composite Aggregation

HeteFedRec: Federated Recommender Systems with Model Heterogeneity

Towards Communication Efficient and Fair Federated Personalized Sequential Recommendation

A split-federated learning and edge-cloud based efficient and privacy-preserving large-scale item recommendation model

Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems

FedPDD: A Privacy-preserving Double Distillation Framework for Cross-silo Federated Recommendation

Federated recommenders: methods, challenges and future

Federated Unlearning for On-Device Recommendation

Representation Learning in Low-rank Slate-based Recommender Systems