Abstract:In a multi-agent system, agents share their local observations to gain global situational awareness for decision making and collaboration using a message passing system. When to send a message, how to encode a message, and how to leverage the received messages directly affect the effectiveness of the collaboration among agents. When training a multi-agent cooperative game using reinforcement learning (RL), the message passing system needs to be optimized together with the agent policies. This consequently increases the model's complexity and poses significant challenges to the convergence and performance of learning. To address this issue, we propose the Belief-map Assisted Multi-agent System (BAMS), which leverages a neuro-symbolic belief map to enhance training. The belief map decodes the agent's hidden state to provide a symbolic representation of the agent's understanding of the environment and other agent's status. The simplicity of symbolic representation allows the gathering and comparison of the ground truth information with the belief, which provides an additional channel of feedback for the learning. Compared to the sporadic and delayed feedback coming from the reward in RL, the feedback from the belief map is more consistent and reliable. Agents using BAMS can learn a more effective message passing network to better understand each other, resulting in better performance in a cooperative predator and prey game with varying levels of map complexity and compare it to previous multi-agent message passing models. The simulation results showed that BAMS reduced training epochs by 66\%, and agents who apply the BAMS model completed the game with 34.62\% fewer steps on average.

Multi-agent Reinforcement Learning with Biased Experience Sharing in Swarm-robotics Domain.

S2rl

A Multi-agent Reinforcement Learning Method for Swarm Robots in Space Collaborative Exploration

Cooperative and Competitive Biases for Multi-Agent Reinforcement Learning

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

Multiexperience-Assisted Efficient Multiagent Reinforcement Learning

Multi-agent Cooperative Games Using Belief Map Assisted Training

Multi-robot Cooperative Navigation Method based on Multi-agent Reinforcement Learning in Sparse Reward Tasks

Hierarchical Consensus-Based Multi-Agent Reinforcement Learning for Multi-Robot Cooperation Tasks

Knowledge Reuse of Multi-Agent Reinforcement Learning in Cooperative Tasks

CRMRL: Collaborative Relationship Meta Reinforcement Learning for Effectively Adapting to Type Changes in Multi-Robotic System

NeuronsMAE: A Novel Multi-Agent Reinforcement Learning Environment for Cooperative and Competitive Multi-Robot Tasks

Inducing Cooperation via Team Regret Minimization based Multi-Agent Deep Reinforcement Learning

A CBR-Based Multiagent Reinforcement Learning Approach

Multi-Robot Cooperative Socially-Aware Navigation Using Multi-Agent Reinforcement Learning

Improving multi-UAV cooperative path-finding through multiagent experience learning

Enabling Multi-Agent Transfer Reinforcement Learning via Scenario Independent Representation

From Multi-agent to Multi-robot: A Scalable Training and Evaluation Platform for Multi-robot Reinforcement Learning

Heterogeneous Multi-Robot Cooperation With Asynchronous Multi-Agent Reinforcement Learning

Autonomous Swarm Robot Coordination via Mean-Field Control Embedding Multi-Agent Reinforcement Learning

On Solving Cooperative MARL Problems with a Few Good Experiences