Abstract:Recently, multiagent reinforcement learning (MARL) has shown great potential for learning cooperative policies in multiagent systems (MASs). However, a noticeable drawback of current MARL is the low sample efficiency, which causes a huge amount of interactions with environment. Such amount of interactions greatly hinders the real-world application of MARL. Fortunately, effectively incorporating experience knowledge can assist MARL to quickly find effective solutions, which can significantly alleviate the drawback. In this article, a novel multiexperience-assisted reinforcement learning (MEARL) method is proposed to improve the learning efficiency of MASs. Specifically, monotonicity-constrained reward shaping is innovatively designed using expert experience to provide additional individual rewards to guide multiagent learning efficiently, with the invariance guarantee of the team optimization objective. Furthermore, a reward distribution estimator is specially developed to model an implicated reward distribution of environment by using transition experience from environment, containing collected samples (state-action pair, reward, and next state). This estimator can predict the expectation reward of each agent for the taken action to accurately estimate the state value function and accelerate its convergence. Besides, the performance of MEARL is evaluated on two multiagent environment platforms: our designed unmanned aerial vehicle combat (UAV-C) and StarCraft II Micromanagement (SCII-M). Simulation results demonstrate that the proposed MEARL can greatly improve the learning efficiency and performance of MASs and is superior to the state-of-the-art methods in multiagent tasks.

Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospects

Observer-Based Multiagent Deep Reinforcement Learning: A Fully Distributed Training Scheme

A review of research on reinforcement learning algorithms for multi-agents

Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges

Review of the Progress of Communication-Based Multi-Agent Reinforcement Learning

A Survey of Progress on Cooperative Multi-agent Reinforcement Learning in Open Environment

Multi-agent Reinforcement Learning: A Comprehensive Survey

Decentralized Multi-Agent Reinforcement Learning with Networked Agents: Recent Advances

Efficient Multi-agent Reinforcement Learning by Planning

Towards Efficient Multi-Agent Learning Systems

LMRL: a Multi-Agent Reinforcement Learning Model and Algorithm

An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective

Model-based Multi-agent Policy Optimization with Adaptive Opponent-wise Rollouts

Multiagent Reinforcement Learning for Strictly Constrained Tasks Based on Reward Recorder

Three perspectives on multi-agent reinforcement learning

An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobility

Multiexperience-Assisted Efficient Multiagent Reinforcement Learning

Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey

Multi-Agent Reinforcement Learning via Adaptive Kalman Temporal Difference and Successor Representation

Multi-Agent Reinforcement Learning for Connected and Automated Vehicles Control: Recent Advancements and Future Prospects

Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computation