Abstract:This paper considers a multi-armed bandit (MAB) problem in which multiple mobile agents receive rewards by sampling from a collection of spatially dispersed stochastic processes, called bandits. The goal is to formulate a decentralized policy for each agent, in order to maximize the total cumulative reward over all agents, subject to option availability and inter-agent communication constraints. The problem formulation is motivated by applications in which a team of autonomous mobile robots cooperates to accomplish an exploration and exploitation task in an uncertain environment. Bandit locations are represented by vertices of the spatial graph. At any time, an agent's option consist of sampling the bandit at its current location, or traveling along an edge of the spatial graph to a new bandit location. Communication constraints are described by a directed, non-stationary, stochastic communication graph. At any time, agents may receive data only from their communication graph in-neighbors. For the case of a single agent on a fully connected spatial graph, it is known that the expected regret for any optimal policy is necessarily bounded below by a function that grows as the logarithm of time. A class of policies called upper confidence bound (UCB) algorithms asymptotically achieve logarithmic regret for the classical MAB problem. In this paper, we propose a UCB-based decentralized motion and option selection policy and a non-stationary stochastic communication protocol that guarantee logarithmic regret. To our knowledge, this is the first such decentralized policy for non-fully connected spatial graphs with communication constraints. When the spatial graph is fully connected and the communication graph is stationary, our decentralized algorithm matches or exceeds the best reported prior results from the literature.

Policies for Risk-Aware Sensor Data Collection by Mobile Agents.

Risk-Aware Planning for Sensor Data Collection

On Optimal Information Capture by Energy-Constrained Mobile Sensors

Desperate Times Call for Desperate Measures: Towards Risk-Adaptive Task Allocation

Incentivizing Mobility of Multi-agent Vehicle Swarms with Deep Reinforcement Learning for Sensing Coverage Optimization.

Mobility and Intruder Prior Information Improving the Barrier Coverage of Sparse Sensor Networks

A Decentralized Policy with Logarithmic Regret for a Class of Multi-Agent Multi-Armed Bandit Problems with Option Unavailability Constraints and Stochastic Communication Protocols

Entropic Risk Measure in Policy Search

Near Optimal Data Gathering in Rechargeable Sensor Networks with a Mobile Sink

Risk-Aware Submodular Optimization for Multi-Robot Coordination

A Game-Theoretic Framework for Managing Risk in Multi-Agent Systems

Risk-Averse Biased Human Policies in Assistive Multi-Armed Bandit Settings

Risk-aware Safe Control for Decentralized Multi-agent Systems via Dynamic Responsibility Allocation

A multiagent urban traffic simulation. Part II: dealing with the extraordinary

Mitigating Side Effects in Multi-Agent Systems Using Blame Assignment

Multi-agent mobile crowdsensing by pervasive machines: a robust task allocation approach

Evolutionary optimization for risk-aware heterogeneous multi-agent path planning in uncertain environments

On Collaboration in Distributed Parameter Estimation with Resource Constraints

Robust Reward Placement under Uncertainty

Building Safer Autonomous Agents by Leveraging Risky Driving Behavior Knowledge

Maximizing Clearance Rate by Penalizing Redundant Task Assignment in Mobile Crowdsensing Auctions