Abstract:Decentralized multi-agent control has broad applications, ranging from multi-robot cooperation to distributed sensor networks. In decentralized multi-agent control, systems are complex with unknown or highly uncertain dynamics, where traditional model-based control methods can hardly be applied. Compared with model-based control in control theory, deep reinforcement learning (DRL) is promising to learn the controller/policy from data without the knowing system dynamics. However, to directly apply DRL to decentralized multi-agent control is challenging, as interactions among agents make the learning environment non-stationary. More importantly, the existing multi-agent reinforcement learning (MARL) algorithms cannot ensure the closed-loop stability of a multi-agent system from a control-theoretic perspective, so the learned control polices are highly possible to generate abnormal or dangerous behaviors in real applications. Hence, without stability guarantee, the application of the existing MARL algorithms to real multi-agent systems is of great concern, e.g., UAVs, robots, and power systems, etc. In this paper, we aim to propose a new MARL algorithm for decentralized multi-agent control with a stability guarantee. The new MARL algorithm, termed as a multi-agent soft-actor critic (MASAC), is proposed under the well-known framework of "centralized-training-with-decentralized-execution". The closed-loop stability is guaranteed by the introduction of a stability constraint during the policy improvement in our MASAC algorithm. The stability constraint is designed based on Lyapunov's method in control theory. To demonstrate the effectiveness, we present a multi-agent navigation example to show the efficiency of the proposed MASAC algorithm.

Multi-agent reinforcement learning for character control

Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges

Multi-agent Reinforcement Learning: A Comprehensive Survey

Overview on reinforcement learning of multi-agent game

A Survey on Reinforcement Learning Methods in Character Animation

Safe Multi-Agent Reinforcement Learning for Multi-Robot Control

A Survey of Progress on Cooperative Multi-agent Reinforcement Learning in Open Environment

Relative Distributed Formation and Obstacle Avoidance with Multi-agent Reinforcement Learning

A review of research on reinforcement learning algorithms for multi-agents

Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey

An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective

Towards Efficient Multi-Agent Learning Systems

Multi-Agent Reinforcement Learning for Connected and Automated Vehicles Control: Recent Advancements and Future Prospects

Deep multiagent reinforcement learning: challenges and directions

A Deep Ensemble Multi-Agent Reinforcement Learning Approach for Air Traffic Control

Review of the Progress of Communication-Based Multi-Agent Reinforcement Learning

Lyapunov-Based Reinforcement Learning for Decentralized Multi-Agent Control

Autonomous and cooperative control of UAV cluster with multi-agent reinforcement learning

Multi-agent Reinforcement Learning with Multi-head Attention

Multi-Agent Reinforcement Learning: A Report on Challenges and Approaches

Revisiting the Master-Slave Architecture in Multi-Agent Deep Reinforcement Learning