Abstract:Reinforcement learning (RL) has been widely applied in recommendation systems due to its potential in optimizing the long-term engagement of users. From the perspective of RL, recommendation can be formulated as a Markov decision process (MDP), where recommendation system (agent) can interact with users (environment) and acquire feedback (reward signals).However, it is impractical to conduct online interactions with the concern on user experience and implementation complexity, and we can only train RL recommenders with offline datasets containing limited reward signals and state transitions. Therefore, the data sparsity issue of reward signals and state transitions is very severe, while it has long been overlooked by existing RL <a class="link-external link-http" href="http://recommenders.Worse" rel="external noopener nofollow">this http URL</a> still, RL methods learn through the trial-and-error mode, but negative feedback cannot be obtained in implicit feedback recommendation tasks, which aggravates the overestimation problem of offline RL recommender. To address these challenges, we propose a novel RL recommender named model-enhanced contrastive reinforcement learning (MCRL). On the one hand, we learn a value function to estimate the long-term engagement of users, together with a conservative value learning mechanism to alleviate the overestimation <a class="link-external link-http" href="http://problem.On" rel="external noopener nofollow">this http URL</a> the other hand, we construct some positive and negative state-action pairs to model the reward function and state transition function with contrastive learning to exploit the internal structure information of MDP. Experiments demonstrate that the proposed method significantly outperforms existing offline RL and self-supervised RL methods with different representative backbone networks on two real-world datasets.

Rejoinder to'Reinforcement learning behaviors in sponsored search'.

Rejoinder to 'reinforcement Learning Behaviors in Sponsored Search'

Reinforcement Learning Behaviors in Sponsored Search

Deep Reinforcement Learning for Sponsored Search Real-time Bidding

Reinforcement Mechanism Design, with Applications to Dynamic Pricing in Sponsored Search Auctions

Optimizing Search Advertising Strategies: Integrating Reinforcement Learning with Generalized Second-Price Auctions for Enhanced Ad Ranking and Bidding

A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges

Sustainable Online Reinforcement Learning for Auto-bidding

Autor3: Au tomated R eal-time R anking with R einforcement Learning in E-commerce Sponsored Search Advertising

Optimizing Enhanced Cost Per Click Via Reinforcement Learning Without Exploration

AdCraft: An Advanced Reinforcement Learning Benchmark Environment for Search Engine Marketing Optimization

Optimizing Sponsored Search Ranking Strategy by Deep Reinforcement Learning

A Game-theoretic Machine Learning Approach for Revenue Maximization in Sponsored Search

Optimizing Long-term Value for Auction-Based Recommender Systems via On-Policy Reinforcement Learning

Budget Constrained Bidding by Model-free Reinforcement Learning in Display Advertising

Reinforcement Learning to Rank in E-Commerce Search Engine: Formalization, Analysis, and Application

User Behavior Simulation for Search Result Re-ranking

Policy Evaluation and Seeking for Multi-Agent Reinforcement Learning Via Best Response

Modeling Bounded Rationality for Sponsored Search Auctions

Model-enhanced Contrastive Reinforcement Learning for Sequential Recommendation

A survey on model-based reinforcement learning