Abstract:Patrolling strategies primarily deal with minimising the time taken to visit specific locations and cover an area. The use of intelligent agents in patrolling has become beneficial in automation and analysing patterns in patrolling. However, practical scenarios demand these strategies to be adaptive in various conditions and robust against adversaries. Traditional Q-learning based patrolling keeps track of all possible states and actions in a Q-table, making them susceptible to the curse of dimensionality. For multi-agent patrolling to be adaptive in various scenarios represented using graphs, we propose a formulation of the Markov Decision Process (MDP) with state-representations that can be utilised for Deep Reinforcement Learning (DRL) approaches such as Deep Q-Networks (DQN). The implemented DQN can estimate the MDP using a finite length state vector trained with a novel reward function. Proposed state-space representation is independent of the number of nodes in the graph, thereby addressing scalability to graph dimensions. We also propose a reward function to penalise the agents for lack of global coordination while providing immediate local feedback on their actions. As independent policy learners subject to the MDP and reward function, the DRL agents formed a collaborative patrolling strategy. The policies learned by the agents generalise and adapt to multiple behaviours without explicit training or design to do so. We provide empirical analysis that shows the strategy’s adaptive capabilities with changes in agents’ position, non-uniform node visit frequency requirements, changes in a graph structure representing the environment, and induced randomness in the trajectories. DRL patrolling proves to be a promising patrolling strategy for intelligent agents by potentially being scalable, adaptive, and robust against adversaries.

Resilient multi-agent RL: introducing DQ-RTS for distributed environments with data loss

Robot Simulation and Reinforcement Learning Training Platform Based on Distributed Architecture.

Target-Value-Competition-Based Multi-Agent Deep Reinforcement Learning Algorithm for Distributed Nonconvex Economic Dispatch

Design and Development of Multi-Agent Reinforcement Learning Intelligence on the Robotarium Platform for Embedded System Applications

Multi-agent Robust Time Differential Reinforcement Learning over Communicated Networks

Deep Reinforcement Learning for Decentralized Multi-Robot Control: A DQN Approach to Robustness and Information Integration

Towards Resilience for Multi-Agent $QD$-Learning

Decentralized Multi-Agent Reinforcement Learning with Global State Prediction

Multi-UAV Redeployment Optimization Based on Multi-Agent Deep Reinforcement Learning Oriented to Swarm Performance Restoration

DQN as an alternative to Market-based approaches for Multi-Robot processing Task Allocation (MRpTA)

Data-Driven Distributed Common Operational Picture from Heterogeneous Platforms using Multi-Agent Reinforcement Learning

Multi-Agent DRL for Distributed Routing and Data Scheduling in Interplanetary Networks

Multi-agent Deep Reinforcement Learning for Resilience-Driven Routing and Scheduling of Mobile Energy Storage Systems

QDAP: Downsizing adaptive policy for cooperative multi-agent reinforcement learning

Multi-robot Cooperative Object Transportation using Decentralized Deep Reinforcement Learning

Conservative and Risk-Aware Offline Multi-Agent Reinforcement Learning for Digital Twins

Decentralized Multi-Robot Formation Control Using Reinforcement Learning

A deep reinforcement learning approach for multi-agent mobile robot patrolling

Traffic Priority-Aware Multi-User Distributed Dynamic Spectrum Access: A Multi-Agent Deep RL Approach

Heterogeneous Multi-Robot Cooperation With Asynchronous Multi-Agent Reinforcement Learning

Distributed Deep Reinforcement Learning: A Survey and a Multi-player Multi-agent Learning Toolbox