Abstract:Recommending novel content, which expands user horizons by introducing them to new interests, has been shown to improve users' long-term experience on recommendation platforms \cite{chen2021values}. Users however are not constantly looking to explore novel content. It is therefore crucial to understand their novelty-seeking intent and adjust the recommendation policy accordingly. Most existing literature models a user's propensity to choose novel content or to prefer a more diverse set of recommendations at individual interactions. Hierarchical structure, on the other hand, exists in a user's novelty-seeking intent, which is manifested as a static and intrinsic user preference for seeking novelty along with a dynamic session-based propensity. To this end, we propose a novel hierarchical reinforcement learning-based method to model the hierarchical user novelty-seeking intent, and to adapt the recommendation policy accordingly based on the extracted user novelty-seeking propensity. We further incorporate diversity and novelty-related measurement in the reward function of the hierarchical RL (HRL) agent to encourage user exploration \cite{chen2021values}. We demonstrate the benefits of explicitly modeling hierarchical user novelty-seeking intent in recommendations through extensive experiments on simulated and real-world datasets. In particular, we demonstrate that the effectiveness of our proposed hierarchical RL-based method lies in its ability to capture such hierarchically-structured intent. As a result, the proposed HRL model achieves superior performance on several public datasets, compared with state-of-art baselines.

Curiosity-driven recommendation strategy for adaptive learning via deep reinforcement learning

Curiosity-Driven Recommendation Strategy for Adaptive Learning via Deep Reinforcement Learning

Adaptive Learning Recommendation Strategy Based on Deep Q-learning

Curiosity-driven Exploration by Self-supervised Prediction

Random curiosity-driven exploration in deep reinforcement learning

Attention-based Curiosity-driven Exploration in Deep Reinforcement Learning

Curiosity-inspired Personalized Recommendation

A unified strategy for implementing curiosity and empowerment driven reinforcement learning

AdaRec: Adaptive Sequential Recommendation for Reinforcing Long-term User Engagement

Dynamic Memory-based Curiosity: A Bootstrap Approach for Exploration

Hierarchical Reinforcement Learning for Modeling User Novelty-Seeking Intent in Recommender Systems

Large-Scale Study of Curiosity-Driven Learning

Intrinsically Motivated Reinforcement Learning Based Recommendation with Counterfactual Data Augmentation

Unified Curiosity-Driven Learning with Smoothed Intrinsic Reward Estimation.

Adaptive Neural Architectures for Recommender Systems

Generative Inverse Deep Reinforcement Learning for Online Recommendation

Self-Attention-Based Temporary Curiosity in Reinforcement Learning Exploration

Deep Reinforcement Learning for Personalized Search Story Recommendation

Give Me Something Unknown: Incorporate Exploration Preference In Cognition Into Recommender System

Deep Exploration for Recommendation Systems

Generative Adversarial User Model for Reinforcement Learning Based Recommendation System