Abstract:Partially observable stochastic games (POSGs) provide a powerful framework for modeling multi-agent interactions. While elegant and expressive, the framework has been shown to be computationally intractable [Bernstein et al., 2002]. An exact dynamic programming algorithm for POSGs has been developed recently, but due to high computational demands, it has only been demonstrated to work on extremely small problems. Several approximate approaches have been developed [Emery-Montemerlo et al., 2004; Nair et al., 2003], but they lack strong theoretical guarantees. In light of these theoretical and practical limitations, there is a need to identify special classes of POSGs that can be solved tractably. One dimension along which computational gain can be leveraged is by exploiting the independence present in the problem dynamics. In this paper, we examine a class of POSGs where the agents only interact loosely by restricting one another’s available actions. Specifically, rather than having fixed action sets, each agent’s action set is a function of the global state. The agents are independent from one another otherwise, i.e. they have separate transition and reward functions that do not interact. This class of problems arises frequently in practice. Many real world domains are inhabited by self-interested agents that act to achieve their individual goals. They may not affect each other in any way, except for occasionally putting restrictions on what each other can do. The best-known solution concepts in game theory are that of computing Nash equilibrium and performing strategy elimination. Our work addresses both of these solution concepts. First, we show that finding a Nash equilibrium where each agent achieves reward at least k is NP-hard. Another contribution of this work is that we present an exact algorithm for performing iterated elimination of dominated strategies. It is able to solve much larger problems than exact algorithms for the general class by exploiting problem structure.

Bridging the Gap between Partially Observable Stochastic Games and Sparse POMDP Methods

Planning for Weakly-Coupled Partially Observable Stochastic Games

A Framework for Sequential Planning in Multi-Agent Settings

PODDP: Partially Observable Differential Dynamic Programming for Latent Belief Space Planning

Partially Observable Multi-Agent Reinforcement Learning with Information Sharing

Decentralized Control of Partially Observable Markov Decision Processes using Belief Space Macro-actions

Rethinking Formal Models of Partially Observable Multiagent Decision Making

Optimality Guarantees for Particle Belief Approximation of POMDPs

Online algorithms for POMDPs with continuous state, action, and observation spaces

Stochastic Dynamic Games in Belief Space

Decentralized control of multi-robot partially observable Markov decision processes using belief space macro-actions

Control Theory Meets POMDPs: A Hybrid Systems Approach

Imprecise Probabilities Meet Partial Observability: Game Semantics for Robust POMDPs

Efficient Multiagent Planning via Shared Action Suggestions

Improving POMDP Tractability Via Belief Compression and Clustering

Monte Carlo Sampling Methods for Approximating Interactive POMDPs

Adaptive Online Packing-guided Search for POMDPs

Learning for Decentralized Control of Multiagent Systems in Large, Partially-Observable Stochastic Environments

Planning with Partially Observable Markov Decision Processes: Advances in Exact Solution Method

Sparse tree search optimality guarantees in POMDPs with continuous observation spaces

Toward Multi-Target Self-Organizing Pursuit in a Partially Observable Markov Game