Abstract:We introduce a stochastic principal-agent model. A principal and an agent interact in a stochastic environment, each privy to observations about the state not available to the other. The principal has the power of commitment, both to elicit information from the agent and to provide signals about her own information. The players communicate with each other and then select actions independently. Each of them receives a payoff based on the state and their joint action, and the environment transitions to a new state. The interaction continues over a finite time horizon. Both players are far-sighted, aiming to maximize their total payoffs over the time horizon. The model encompasses as special cases extensive-form games (EFGs) and stochastic games of incomplete information, partially observable Markov decision processes (POMDPs), as well as other forms of sequential principal-agent interactions, including Bayesian persuasion and automated mechanism design problems. We consider both the computation and learning of the principal's optimal policy. Since the general problem, which subsumes POMDPs, is intractable, we explore algorithmic solutions under hindsight observability, where the state and the interaction history are revealed at the end of each step. Though the problem becomes more amenable under this condition, the number of possible histories remains exponential in the length of the time horizon, making approaches for EFG-based models infeasible. We present an efficient algorithm based on the inducible value sets. The algorithm computes an $\epsilon$-approximate optimal policy in time polynomial in $1/\epsilon$. Additionally, we show an efficient learning algorithm for an episodic reinforcement learning setting where the transition probabilities are unknown. The algorithm guarantees sublinear regret $\tilde{O}(T^{2/3})$ for both players over $T$ episodes.

Kolkata Paise Restaurant Problem in Some Uniform Learning Strategy Limits

The Kolkata Paise Restaurant Problem and Resource Utilization

Phase transition in the Kolkata Paise Restaurant problem

Stochastic Learning in Kolkata Paise Restaurant Problem: Classical \& Quantum Strategies

DKPRG or how to succeed in the Kolkata Paise Restaurant gamevia TSP

Stability of Dining Clubs in the Kolkata Paise Problem with and without Cheating

Emergence of Distributed Coordination in the Kolkata Paise Restaurant Problem with Finite Information

Online Learning and Pricing for Multiple Products with Reference Price Effects

Indian Buffet Game with Negative Network Externality and Non-Bayesian Social Learning.

The Saga of KPR: Theoretical and Experimental developments

Indian Buffet Game with non-Bayesian social learning.

Chinese Restaurant Game - Part I: Theory of Learning with Negative Network Externality

Dynamic and non-uniform pricing strategies for revenue maximization

Earning and Learning with Varying Cost

The equivalence of dynamic and strategic stability under regularized learning in games

Think Before You Duel: Understanding Complexities of Preference Learning under Constrained Resources

Stochastic Principal-Agent Problems: Efficient Computation and Learning

Sequential Chinese Restaurant Game

Learning to Control Unknown Strongly Monotone Games

Optimal Pricing Schemes in the Presence of Social Learning and Costly Reporting

Maximum a posteriori learning in demand competition games