Abstract:Online learning to rank (ONL2R) is a foundational problem for recommender systems and has received increasing attention in recent years. Among the existing approaches for ONL2R, a natural modeling architecture is the multi-armed bandit framework coupled with the position-based click model. However, developing efficient online learning policies for MAB-based ONL2R with position-based click models is highly challenging due to the combinatorial nature of the problem, and partial observability in the position-based click model. To date, results in MAB-based ONL2R with position-based click models remain rather limited, which motivates us to fill this gap in this work. Our main contributions in this work are threefold: i) We propose the first general MAB framework that captures all key ingredients of ONL2R with position-based click models. Our model considers personalized and equal treatments in ONL2R ranking recommendations, both of which are widely used in practice; ii) Based on the above analytical framework, we develop two unified greed- and UCB-based policies called GreedyRank and UCBRank, each of which can be applied to personalized and equal ranking treatments; and iii) We show that both GreedyRank and UCBRank enjoy $O(\sqrt{t}\ln t)$ and $O(\sqrt{t\ln t})$ anytime sublinear regret for personalized and equal treatment, respectively. For the fundamentally hard equal ranking treatment, we identify classes of collective utility functions and their associated sufficient conditions under which $O(\sqrt{t}\ln t)$ and $O(\sqrt{t\ln t})$ anytime sublinear regrets are still achievable for GreedyRank and UCBRank, respectively. Our numerical experiments also verify our theoretical results and demonstrate the efficiency of GreedyRank and UCBRank in seeking the optimal action under various problem settings.

Personalized Advertisement Recommendation: A Ranking Approach to Address the Ubiquitous Click Sparsity Problem

A Contextual-Bandit Approach to Personalized News Article Recommendation

Self-Attention Mechanism Enhanced User Interests Modeling for Personalized Recommendation Services in Cyber-Physical-Social Systems.

Context-aware advertisement recommendation for high-speed social news feeding

Rankmbpr: Rank-Aware Mutual Bayesian Personalized Ranking For Item Recommendationl

Bandit Learning to Rank with Position-Based Click Models: Personalized and Equal Treatments

Understanding the Ranking Loss for Recommendation with Sparse User Feedback

Dimensionality reduction for click-through rate prediction: Dense versus sparse representation

Learning Personalized Page Content Ranking Using Customer Representation

A Contextual Bandit Approach to Personalized Online Recommendation via Sparse Interactions.

A Bird's-eye View of Reranking: from List Level to Page Level

Adversarial Personalized Ranking for Recommendation

Fatigue-aware Bandits for Dependent Click Models

Clicks can be Cheating: Counterfactual Recommendation for Mitigating Clickbait Issue

Top-Personalized-K Recommendation

Learning to Rank Audience for Behavioral Targeting in Display Ads

Maximizing Cumulative User Engagement in Sequential Recommendation: An Online Optimization Perspective

Contextual Bandit Approach-based Recommendation System for Personalized Web-based Services

Pessimistic Off-Policy Optimization for Learning to Rank

Cross Pairwise Ranking for Unbiased Item Recommendation

Recall-Augmented Ranking: Enhancing Click-Through Rate Prediction Accuracy with Cross-Stage Data