Abstract:Partial monitoring games are repeated games where the learner receives feedback that might be different from adversary's move or even the reward gained by the learner. Recently, a general model of combinatorial partial monitoring (CPM) games was proposed \cite{lincombinatorial2014}, where the learner's action space can be exponentially large and adversary samples its moves from a bounded, continuous space, according to a fixed distribution. The paper gave a confidence bound based algorithm (GCB) that achieves $O(T^{2/3}\log T)$ distribution independent and $O(\log T)$ distribution dependent regret bounds. The implementation of their algorithm depends on two separate offline oracles and the distribution dependent regret additionally requires existence of a unique optimal action for the learner. Adopting their CPM model, our first contribution is a Phased Exploration with Greedy Exploitation (PEGE) algorithmic framework for the problem. Different algorithms within the framework achieve $O(T^{2/3}\sqrt{\log T})$ distribution independent and $O(\log^2 T)$ distribution dependent regret respectively. Crucially, our framework needs only the simpler "argmax" oracle from GCB and the distribution dependent regret does not require existence of a unique optimal action. Our second contribution is another algorithm, PEGE2, which combines gap estimation with a PEGE algorithm, to achieve an $O(\log T)$ regret bound, matching the GCB guarantee but removing the dependence on size of the learner's action space. However, like GCB, PEGE2 requires access to both offline oracles and the existence of a unique optimal action. Finally, we discuss how our algorithm can be efficiently applied to a CPM problem of practical interest: namely, online ranking with feedback at the top.

A PDE Approach for Regret Bounds under Partial Monitoring

Minimax Regret for Partial Monitoring: Infinite Outcomes and Rustichini's Regret

Randomized Confidence Bounds for Stochastic Partial Monitoring

Toward a Classification of Finite Partial-Monitoring Games

Linear Partial Monitoring for Sequential Decision-Making: Algorithms, Regret Bounds and Applications

Forecasting for Swap Regret for All Downstream Agents

No Internal Regret via Neighborhood Watch

Second Order Regret Bounds Against Generalized Expert Sequences under Partial Bandit Feedback

Best-of-Both-Worlds Algorithms for Partial Monitoring

Provably Efficient Partially Observable Risk-Sensitive Reinforcement Learning with Hindsight Observation

Refined Regret for Adversarial MDPs with Linear Function Approximation

Phased Exploration with Greedy Exploitation in Stochastic Combinatorial Partial Monitoring Games

Optimistic Posterior Sampling for Reinforcement Learning: Worst-Case Regret Bounds

Analysis and Design of Thompson Sampling for Stochastic Partial Monitoring

A Simple Approach For Non-Stationary Linear Bandits

Sub-linear Regret in Adaptive Model Predictive Control

Horizon-Free and Variance-Dependent Reinforcement Learning for Latent Markov Decision Processes

$\Sqrt{n}$-Regret for Learning in Markov Decision Processes with Function Approximation and Low Bellman Rank

Online Linear Regression in Dynamic Environments via Discounting

Calibration and Internal no-Regret with Partial Monitoring

Exploration by Optimization with Hybrid Regularizers: Logarithmic Regret with Adversarial Robustness in Partial Monitoring