Abstract:Multi-agent reinforcement learning (MARL) has been increasingly explored to learn the cooperative policy towards maximizing a certain global reward. Many existing studies take advantage of graph neural networks (GNN) in MARL to propagate critical collaborative information over the interaction graph, built upon inter-connected agents. Nevertheless, the vanilla GNN approach yields substantial defects in dealing with complex real-world scenarios since the generic message passing mechanism is ineffective between heterogeneous vertices and, moreover, simple message aggregation functions are incapable of accurately modeling the combinational interactions from multiple neighbors. While adopting complex GNN models with more informative message passing and aggregation mechanisms can obviously benefit heterogeneous vertex representations and cooperative policy learning, it could, on the other hand, increase the training difficulty of MARL and demand more intense and direct reward signals compared to the original global reward. To address these challenges, we propose a new cooperative learning framework with pre-trained heterogeneous observation representations. Particularly, we employ an encoder-decoder based graph attention to learn the intricate interactions and heterogeneous representations that can be more easily leveraged by MARL. Moreover, we design a pre-training with local actor-critic algorithm to ease the difficulty in cooperative policy learning. Extensive experiments over real-world scenarios demonstrate that our new approach can significantly outperform existing MARL baselines as well as operational research solutions that are widely-used in industry.

Graph Neural Network Meets Multi-Agent Reinforcement Learning: Fundamentals, Applications, and Future Directions

Graphcomm: a graph neural network based method for multi-agent reinforcement learning

Multiagent Reinforcement Learning With Graphical Mutual Information Maximization

Robust Multi-Agent Communication With Graph Information Bottleneck Optimization

Graph Neural Network-based Multi-agent Reinforcement Learning for Resilient Distributed Coordination of Multi-Robot Systems

Scaling Up Multiagent Reinforcement Learning for Robotic Systems: Learn an Adaptive Sparse Communication Graph

Graph Neural Networks with Model-based Reinforcement Learning for Multi-agent Systems

HyperComm: Hypergraph-based communication in multi-agent reinforcement learning

Learning Efficient and Robust Multi-Agent Communication Via Graph Information Bottleneck

Networked Multi-Agent Reinforcement Learning with Emergent Communication

A Cooperation Graph Approach for Multiagent Sparse Reward Reinforcement Learning

Graph Neural Networks Meet Wireless Communications: Motivation, Applications, and Future Directions

Multiagent Reinforcement Learning With Heterogeneous Graph Attention Network

Efficient Communications in Multi-Agent Reinforcement Learning for Mobile Applications

GraphComm: Efficient Graph Convolutional Communication for Multiagent Cooperation

Multi-Agent Game Abstraction Via Graph Attention Neural Network.

Decentralized Multi-Agent Reinforcement Learning with Networked Agents: Recent Advances

Heterogeneous Multi-Agent Communication Learning Via Graph Information Maximization.

Cooperative Policy Learning with Pre-trained Heterogeneous Observation Representations

Graph Neural Networks for Intelligent Modelling in Network Management and Orchestration: A Survey on Communications

Graph Neural Networks for Wireless Networks: Graph Representation, Architecture and Evaluation