Abstract:One of the challenges for multi-agent reinforcement learning (MARL) is designing efficient learning algorithms for a large system in which each agent has only limited or partial information of the entire system. In this system, it is desirable to learn policies of a decentralized type. A recent and promising paradigm to analyze such decentralized MARL is to take network structures into consideration. While exciting progress has been made to analyze decentralized MARL with the network of agents, often found in social networks and team video games, little is known theoretically for decentralized MARL with the network of states, frequently used for modeling self-driving vehicles, ride-sharing, and data and traffic routing.This paper proposes a framework called localized training and decentralized execution to study MARL with network of states, with homogeneous (a.k.a. mean-field type) agents. Localized training means that agents only need to collect local information in their neighboring states during the training phase; decentralized execution implies that, after the training stage, agents can execute the learned decentralized policies, which only requires knowledge of the agents’ current states. The key idea is to utilize the homogeneity of agents and regroup them according to their states, thus the formulation of a networked Markov decision process with teams of agents, enabling the update of the Q-function in a localized fashion. In order to design an efficient and scalable reinforcement learning algorithm under such a framework, we adopt the actor-critic approach with over-parameterized neural networks, and establish the convergence and sample complexity for our algorithm, shown to be scalable with respect to the size of both agents and states.

Multi-Agent Reinforcement Learning for Network Load Balancing in Data Center

Scalable Multi-Agent Reinforcement Learning for Residential Load Scheduling under Data Governance

Multi-agent Actor-Critic Reinforcement Learning Based In-network Load Balance.

Load Balancing for Task Scheduling Based on Multi-Agent Reinforcement Learning in Cloud-Edge-End Collaborative Environments

Multi-agent Reinforcement Learning for Dynamic Dispatching in Material Handling Systems

Multi-Agent Reinforcement Learning for Wireless User Scheduling: Performance, Scalablility, and Generalization

A Cooperative Multi-Agent Reinforcement Learning Framework for Resource Balancing in Complex Logistics Network

MARVEL: Enabling controller load balancing in software-defined networks with multi-agent reinforcement learning

On Traffic Routing in Network Powered by Computing Environment: ECMP vs. UCMP vs. Multi-Agent MAROH

Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Reward

Multi-Agent Reinforcement Learning for Multi-Cell Spectrum and Power Allocation

Fully Distributed Fog Load Balancing with Multi-Agent Reinforcement Learning

DeepRLB: A deep reinforcement learning‐based load balancing in data center networks

Heterogeneous Multi-Agent Reinforcement Learning for Distributed Channel Access in WLANs

Multi-Dimensional Resource Allocation in Distributed Data Centers Using Deep Reinforcement Learning

Reinforced Workload Distribution Fairness

Mean-Field Multi-Agent Reinforcement Learning: A Decentralized Network Approach

General multi-agent reinforcement learning integrating heuristic-based delay priority strategy for demand and capacity balancing

Sparse Mean Field Load Balancing in Large Localized Queueing Systems

Towards Efficient Multi-Agent Learning Systems