Abstract:Future Internet involves several emerging technologies such as 5G and beyond 5G networks, vehicular networks, unmanned aerial vehicle (UAV) networks, and Internet of Things (IoTs). Moreover, the future Internet becomes heterogeneous and decentralized with a large number of involved network entities. Each entity may need to make its local decision to improve the network performance under dynamic and uncertain network environments. Standard learning algorithms such as single-agent Reinforcement Learning (RL) or Deep Reinforcement Learning (DRL) have been recently used to enable each network entity as an agent to learn an optimal decision-making policy adaptively through interacting with the unknown environments. However, such an algorithm fails to model the cooperations or competitions among network entities, and simply treats other entities as a part of the environment that may result in the non-stationarity issue. Multi-agent Reinforcement Learning (MARL) allows each network entity to learn its optimal policy by observing not only the environments but also other entities' policies. As a result, MARL can significantly improve the learning efficiency of the network entities, and it has been recently used to solve various issues in the emerging networks. In this paper, we thus review the applications of MARL in emerging networks. In particular, we provide a tutorial of MARL and a comprehensive survey of applications of MARL in next-generation Internet. In particular, we first introduce single-agent RL and MARL. Then, we review a number of applications of MARL to solve emerging issues in the future Internet. The issues consist of network access, transmit power control, computation offloading, content caching, packet routing, trajectory design for UAV-aided networks, and network security issues. Finally, we discuss the challenges, open issues, and future directions related to the applications of MARL in the future Internet.

A Comprehensive Survey on Multi-Agent Reinforcement Learning for Connected and Automated Vehicles

Multi-Agent Reinforcement Learning for Connected and Automated Vehicles Control: Recent Advancements and Future Prospects

Multi-agent reinforcement learning for autonomous vehicles: a survey

An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobility

Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey

A Multi-Agent Reinforcement Learning Approach For Safe and Efficient Behavior Planning Of Connected Autonomous Vehicles

A Survey of Progress on Cooperative Multi-agent Reinforcement Learning in Open Environment

Multi-agent Reinforcement Learning: A Comprehensive Survey

Communication-Efficient Decentralized Multi-Agent Reinforcement Learning for Cooperative Adaptive Cruise Control

Multi-Intersection Management for Connected Autonomous Vehicles by Reinforcement Learning

Shared Information-Based Safe And Efficient Behavior Planning For Connected Autonomous Vehicles

Autonomous Intersection Management with Heterogeneous Vehicles: A Multi-Agent Reinforcement Learning Approach

MARP: A Cooperative Multiagent DRL System for Connected Autonomous Vehicle Platooning

Multi-agent reinforcement learning for cooperative lane changing of connected and autonomous vehicles in mixed traffic

Safety Guaranteed Robust Multi-Agent Reinforcement Learning with Hierarchical Control for Connected and Automated Vehicles

Enforcing Cooperative Safety for Reinforcement Learning-based Mixed-Autonomy Platoon Control

A Multi-Agent Deep Reinforcement Learning Coordination Framework for Connected and Automated Vehicles at Merging Roadways

A review of research on reinforcement learning algorithms for multi-agents

Communication-Aware Reinforcement Learning for Cooperative Adaptive Cruise Control

Multi-Vehicle Cooperative Decision-Making in Merging Area Based on Deep Multi-Agent Reinforcement Learning

Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey