Abstract:Patrolling strategies primarily deal with minimising the time taken to visit specific locations and cover an area. The use of intelligent agents in patrolling has become beneficial in automation and analysing patterns in patrolling. However, practical scenarios demand these strategies to be adaptive in various conditions and robust against adversaries. Traditional Q-learning based patrolling keeps track of all possible states and actions in a Q-table, making them susceptible to the curse of dimensionality. For multi-agent patrolling to be adaptive in various scenarios represented using graphs, we propose a formulation of the Markov Decision Process (MDP) with state-representations that can be utilised for Deep Reinforcement Learning (DRL) approaches such as Deep Q-Networks (DQN). The implemented DQN can estimate the MDP using a finite length state vector trained with a novel reward function. Proposed state-space representation is independent of the number of nodes in the graph, thereby addressing scalability to graph dimensions. We also propose a reward function to penalise the agents for lack of global coordination while providing immediate local feedback on their actions. As independent policy learners subject to the MDP and reward function, the DRL agents formed a collaborative patrolling strategy. The policies learned by the agents generalise and adapt to multiple behaviours without explicit training or design to do so. We provide empirical analysis that shows the strategy’s adaptive capabilities with changes in agents’ position, non-uniform node visit frequency requirements, changes in a graph structure representing the environment, and induced randomness in the trajectories. DRL patrolling proves to be a promising patrolling strategy for intelligent agents by potentially being scalable, adaptive, and robust against adversaries.

Decentralized Exploration of a Structured Environment Based on Multi-agent Deep Reinforcement Learning.

Moving Forward in Formation: A Decentralized Hierarchical Learning Approach to Multi-Agent Moving Together

Observer-Based Multiagent Deep Reinforcement Learning: A Fully Distributed Training Scheme

Mapless Collaborative Navigation for a Multi-Robot System Based on the Deep Reinforcement Learning

Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domain

An Enhanced Hierarchical Planning Framework for Multi-Robot Autonomous Exploration

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

Deep Reinforcement Learning-based Large-scale Robot Exploration

Passivity analysis for switched generalized neural networks with time-varying delay and uncertain output

Real-time local path planning strategy based on deep distributional reinforcement learning

Hybrid Centralized Training and Decentralized Execution Reinforcement Learning in Multi-Agent Path-Finding Simulations

Multi-Agent Exploration of an Unknown Sparse Landmark Complex via Deep Reinforcement Learning

Mars Exploration: Research on Goal-Driven Hierarchical DQN Autonomous Scene Exploration Algorithm

Deep Reinforcement Learning with Multi-Critic TD3 for Decentralized Multi-Robot Path Planning

Distributed Resource Scheduling for Large-Scale MEC Systems: A Multiagent Ensemble Deep Reinforcement Learning With Imitation Acceleration

Deep Reinforcement Learning for Decentralized Multi-Robot Control: A DQN Approach to Robustness and Information Integration

Hierarchical Reinforcement Learning with Opponent Modeling for Distributed Multi-agent Cooperation

Asymmetric Information Enhanced Mapping Framework for Multirobot Exploration based on Deep Reinforcement Learning

Decentralized Strategy for Cooperative Multi-Robot Exploration and Mapping

A deep reinforcement learning approach for multi-agent mobile robot patrolling