Abstract:Many current large-scale multiagent team implementations can be characterized as following the belief-desire-intention (BDI) paradigm, with explicit representation of team plans. Despite their promise, current BDI team approaches lack tools for quantitative performance analysis under uncertainty. Distributed partially observable Markov decision problems (POMDPs) are well suited for such analysis, but the complexity of finding optimal policies in such models is highly intractable. The key contribution of this article is a hybrid BDI-POMDP approach, where BDI team plans are exploited to improve POMDP tractability and POMDP analysis improves BDI team plan performance. Concretely, we focus on role allocation, a fundamental problem in BDI teams: which agents to allocate to the different roles in the team. The article provides three key contributions. First, we describe a role allocation technique that takes into account future uncertainties in the domain; prior work in multiagent role allocation has failed to address such uncertainties. To that end, we introduce RMTDP (Role-based Markov Team Decision Problem), a new distributed POMDP model for analysis of role allocations. Our technique gains in tractability by significantly curtailing RMTDP policy search; in particular, BDI team plans provide incomplete RMTDP policies, and the RMTDP policy search fills the gaps in such incomplete policies by searching for the best role allocation. Our second key contribution is a novel decomposition technique to further improve RMTDP policy search efficiency. Even though limited to searching role allocations, there are still combinatorially many role allocations, and evaluating each in RMTDP to identify the best is extremely difficult. Our decomposition technique exploits the structure in the BDI team plans to significantly prune the search space of role allocations. Our third key contribution is a significantly faster policy evaluation algorithm suited for our BDI-POMDP hybrid approach. Finally, we also present experimental results from two domains: mission rehearsal simulation and RoboCupRescue disaster rescue simulation.

A Role-Based POMDPs Approach for Decentralized Implicit Cooperation of Multiple Agents.

Efficient Multiagent Planning via Shared Action Suggestions

Cooperative Multi-Agent Constrained POMDPs: Strong Duality and Primal-Dual Reinforcement Learning with Approximate Information States

A POMDP Approach to Token-Based Team Coordination

Planning for Decentralized Control of Multiple Robots Under Uncertainty

Hybrid BDI-POMDP Framework for Multiagent Teaming

Decentralized Opportunistic Spectrum Resources Access Model and Algorithm Toward Cooperative Ad-Hoc Networks

Learning for Decentralized Control of Multiagent Systems in Large, Partially-Observable Stochastic Environments

Integrating Decision Sharing with Prediction in Decentralized Planning for Multi-Agent Coordination under Uncertainty.

Decentralized Channel Access Mechanism Design for Multi-robot Coordination

Research on Decentralized Communication Decision in the Multi-Agent Robotic System

Decentralized Role Assignment in Multi-Agent Teams via Empirical Game-Theoretic Analysis

Decentralized Control of Partially Observable Markov Decision Processes using Belief Space Macro-actions

A Novel Optimal Bipartite Consensus Control Scheme for Unknown Multi-Agent Systems Via Model-Free Reinforcement Learning.

Decentralized control of multi-robot partially observable Markov decision processes using belief space macro-actions

Multi-agent Coordination Under Temporal Logic Tasks and Team-Wise Intermittent Communication

Communication Decision in Decentralized Control of Coordinated System

Bridging the Gap between Partially Observable Stochastic Games and Sparse POMDP Methods

Multi-Agent Coordination via Multi-Level Communication

Locally Interdependent Multi-Agent MDP: Theoretical Framework for Decentralized Agents with Dynamic Dependencies

Coordination as inference in multi-agent reinforcement learning