Abstract:Reinforcement learning (RL) has been widely applied in recommendation systems due to its potential in optimizing the long-term engagement of users. From the perspective of RL, recommendation can be formulated as a Markov decision process (MDP), where recommendation system (agent) can interact with users (environment) and acquire feedback (reward signals).However, it is impractical to conduct online interactions with the concern on user experience and implementation complexity, and we can only train RL recommenders with offline datasets containing limited reward signals and state transitions. Therefore, the data sparsity issue of reward signals and state transitions is very severe, while it has long been overlooked by existing RL <a class="link-external link-http" href="http://recommenders.Worse" rel="external noopener nofollow">this http URL</a> still, RL methods learn through the trial-and-error mode, but negative feedback cannot be obtained in implicit feedback recommendation tasks, which aggravates the overestimation problem of offline RL recommender. To address these challenges, we propose a novel RL recommender named model-enhanced contrastive reinforcement learning (MCRL). On the one hand, we learn a value function to estimate the long-term engagement of users, together with a conservative value learning mechanism to alleviate the overestimation <a class="link-external link-http" href="http://problem.On" rel="external noopener nofollow">this http URL</a> the other hand, we construct some positive and negative state-action pairs to model the reward function and state transition function with contrastive learning to exploit the internal structure information of MDP. Experiments demonstrate that the proposed method significantly outperforms existing offline RL and self-supervised RL methods with different representative backbone networks on two real-world datasets.

ReInform: Selecting paths with reinforcement learning for contextualized link prediction

RLPath: a knowledge graph link prediction method using reinforcement learning based attentive relation path searching and representation learning

Model-enhanced Contrastive Reinforcement Learning for Sequential Recommendation

Decision Mamba: Reinforcement Learning via Hybrid Selective Sequence Modeling

In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought

On Transforming Reinforcement Learning by Transformer: The Development Trajectory

Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining

Bi-Link: Bridging Inductive Link Predictions from Text via Contrastive Learning of Transformers and Prompts

Deep Reinforcement Learning For Sequence to Sequence Models

On Transforming Reinforcement Learning With Transformers: The Development Trajectory

Teach and Explore: A Multiplex Information-guided Effective and Efficient Reinforcement Learning for Sequential Recommendation

Decision Transformer: Reinforcement Learning via Sequence Modeling

Retentive Decision Transformer with Adaptive Masking for Reinforcement Learning based Recommendation Systems

RLINK: Deep Reinforcement Learning for User Identity Linkage

Transformers are Meta-Reinforcement Learners

Link Prediction with Non-Contrastive Learning

Reinformer: Max-Return Sequence Modeling for Offline RL

Efficient Recurrent Off-Policy RL Requires a Context-Encoder-Specific Learning Rate

A deep inverse reinforcement learning approach to route choice modeling with context-dependent rewards

Integrating Offline Reinforcement Learning with Transformers for Sequential Recommendation

Contextual Transformer for Offline Meta Reinforcement Learning