Abstract:Many physical systems have underlying safety considerations that require that the strategy deployed ensures the satisfaction of a set of constraints. Further, often we have only partial information on the state of the system. We study the problem of safe real-time decision making under uncertainty. In this paper, we formulate a conservative stochastic contextual bandit formulation for real-time decision making when an adversary chooses a distribution on the set of possible contexts and the learner is subject to certain safety/performance constraints. The learner observes only the context distribution and the exact context is unknown, and the goal is to develop an algorithm that selects a sequence of optimal actions to maximize the cumulative reward without violating the safety constraints at any time step. By leveraging the UCB algorithm for this setting, we propose a conservative linear UCB algorithm for stochastic bandits with context distribution. We prove an upper bound on the regret of the algorithm and show that it can be decomposed into three terms: (i) an upper bound for the regret of the standard linear UCB algorithm, (ii) a constant term (independent of time horizon) that accounts for the loss of being conservative in order to satisfy the safety constraint, and (ii) a constant term (independent of time horizon) that accounts for the loss for the contexts being unknown and only the distribution being known. To validate the performance of our approach we perform extensive simulations on synthetic data and on real-world maize data collected through the Genomes to Fields (G2F) initiative.

Model Selection for Generic Contextual Bandits

Causal Contextual Bandits with Adaptive Context

Estimation Considerations in Contextual Bandits

Stochastic Bandits with Context Distributions

Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizability

A Bayesian Approach for Subset Selection in Contextual Bandits.

Robust Contextual Linear Bandits.

Stochastic Conservative Contextual Linear Bandits

Efficient and Robust High-Dimensional Linear Contextual Bandits.

Contextual Bandits with Stage-wise Constraints

Contextual Bandits for Unbounded Context Distributions

Selectively Contextual Bandits

Simple Regret Minimization for Contextual Bandits

OSOM: A simultaneously optimal algorithm for multi-armed and linear contextual bandits

Contextual Continuum Bandits: Static Versus Dynamic Regret

A Hierarchical Nearest Neighbour Approach to Contextual Bandits

Generalized Linear Bandits with Limited Adaptivity

Instance-Dependent Complexity of Contextual Bandits and Reinforcement Learning: A Disagreement-Based Perspective

Leveraging Good Representations in Linear Contextual Bandits

Smooth Contextual Bandits: Bridging the Parametric and Non-differentiable Regret Regimes

Proportional Response: Contextual Bandits for Simple and Cumulative Regret Minimization