Abstract:Multi-agent reinforcement learning (MARL) is a challenging branch of reinforcement learning that requires cooperation of interactive learning agents to achieve individual objectives as well as shared team objectives. Existing MARL algorithms generally use either centralized global state representation or decentralized local observation to perform training and execution. In this paper, we introduce a novel MARL learning paradigm, centralized training with semi-centralized execution (CTSCE), and present a new MARL algorithm for addressing multi-agent problems: Semi-Centralized Multi-Agent Imitation Reinforcement Learning (SC-MAIRL). The semi-centralized approach aggregated with agents' spatial and temporal information serves as a joint knowledge base to facilitate a learning agent to discover team objectives and make fine-grained decisions. We also utilize a pre-trained performant teacher policy to guide an untrained model towards positive game states as a form of imitation learning, significantly increasing the agent's learning speed. In addition, to encourage agents to learn both offensive and defensive behaviors and smooth the high-dimensional learning curve, we present a new set of reward-shaping functions to further improve SC-MAIRL's learning performance. Our approach is evaluated using one of the most challenging scenarios within the StarCraft Multi-Agent Challenge environment, and the results show that SC-MAIRL outperforms the state-of-the-art MARL algorithm MAPPO in several metrics and allows our agents to learn and employ novel, complex macro strategies more effectively.

A review of research on reinforcement learning algorithms for multi-agents

Multiagent Reinforcement Learning for Strictly Constrained Tasks Based on Reward Recorder

Review of the Progress of Communication-Based Multi-Agent Reinforcement Learning

Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges

A Survey of Progress on Cooperative Multi-agent Reinforcement Learning in Open Environment

Multi-agent Reinforcement Learning: A Comprehensive Survey

Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospects

Decentralized Multi-Agent Reinforcement Learning with Networked Agents: Recent Advances

An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobility

Overview on reinforcement learning of multi-agent game

Towards Efficient Multi-Agent Learning Systems

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective

Multiagent Reinforcement Learning: Methods, Trustworthiness, Applications in Intelligent Vehicles, and Challenges

Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks

Multi-Agent Reinforcement Learning from Human Feedback: Data Coverage and Algorithmic Techniques

Scalability Bottlenecks in Multi-Agent Reinforcement Learning Systems

Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey

A Survey and Critique of Multiagent Deep Reinforcement Learning

LMRL: a Multi-Agent Reinforcement Learning Model and Algorithm

SC-MAIRL: Semi-Centralized Multi-Agent Imitation Reinforcement Learning