Abstract:Multi-agent reinforcement learning (MARL) algorithms have accomplished remarkable breakthroughs in solving large-scale decision-making tasks. Nonetheless, most existing MARL algorithms are model-free, limiting sample efficiency and hindering their applicability in more challenging scenarios. In contrast, model-based reinforcement learning (MBRL), particularly algorithms integrating planning, such as MuZero, has demonstrated superhuman performance with limited data in many tasks. Hence, we aim to boost the sample efficiency of MARL by adopting model-based approaches. However, incorporating planning and search methods into multi-agent systems poses significant challenges. The expansive action space of multi-agent systems often necessitates leveraging the nearly-independent property of agents to accelerate learning. To tackle this issue, we propose the MAZero algorithm, which combines a centralized model with Monte Carlo Tree Search (MCTS) for policy search. We design a novel network structure to facilitate distributed execution and parameter sharing. To enhance search efficiency in deterministic environments with sizable action spaces, we introduce two novel techniques: Optimistic Search Lambda (OS($\lambda$)) and Advantage-Weighted Policy Optimization (AWPO). Extensive experiments on the SMAC benchmark demonstrate that MAZero outperforms model-free approaches in terms of sample efficiency and provides comparable or better performance than existing model-based methods in terms of both sample and computational efficiency. Our code is available at

Planning Rule Extracting Algorithm of Multi-Agent System Based on Reinforcement Learning

A Plan Recognition Approach for Agent in Adversarial Multi-Agent System

The Multi-Agent System Based on Reinforcement Learning

Attention-Cooperated Reinforcement Learning for Multi-agent Path Planning

Multiagent Motion Planning Based on Deep Reinforcement Learning in Complex Environments

Multi-Agent Path Finding Method Based on Evolutionary Reinforcement Learning

An AI Planning-Based Approach to the Multi-Agent Plan Recognition Problem

Multi-domain Cooperative Action Planning Strategy Based on Reinforcement Learning

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

Multi-agent cooperative reinforcement learning algorithm based on practical reasoning

Efficient Multi-agent Reinforcement Learning by Planning

A multi-agent planning approach integrated with learning mechanism

Action-Model Based Multi-agent Plan Recognition

A Novel Multi-Agent Reinforcement Learning Approach

Cooperative Reward Shaping for Multi-Agent Pathfinding

Multi-agent Cooperative Strategy Learning Method Based on Transfer Learning.

Multi-Agent Path Finding Using Imitation-Reinforcement Learning with Transformer

LMRL: a Multi-Agent Reinforcement Learning Model and Algorithm

Learning Efficient Multi-Agent Cooperative Visual Exploration

Reinforcement learning based on multi-agent in robocup

Intent-aware Multi-agent Reinforcement Learning