Abstract:In this study, we examine the problem of downlink wireless routing in integrated access backhaul (IAB) networks involving fiber-connected base stations, wireless base stations, and multiple users. Physical constraints prevent the use of a central controller, leaving base stations with limited access to real-time network conditions. These networks operate in a time-slotted regime, where base stations monitor network conditions and forward packets accordingly. Our objective is to maximize the arrival ratio of packets, while simultaneously minimizing their latency. To accomplish this, we formulate this problem as a multi-agent partially observed Markov Decision Process (POMDP). Moreover, we develop an algorithm that uses Multi-Agent Reinforcement Learning (MARL) combined with Advantage Actor Critic (A2C) to derive a joint routing policy on a distributed basis. Due to the importance of packet destinations for successful routing decisions, we utilize information about similar destinations as a basis for selecting specific-destination routing decisions. For portraying the similarity between those destinations, we rely on their relational base-station associations, i.e., which base station they are currently connected to. Therefore, the algorithm is referred to as Relational Advantage Actor Critic (Relational A2C). To the best of our knowledge, this is the first work that optimizes routing strategy for IAB networks. Further, we present three types of training paradigms for this algorithm in order to provide flexibility in terms of its performance and throughput. Through numerical experiments with different network scenarios, Relational A2C algorithms were demonstrated to be capable of achieving near-centralized performance even though they operate in a decentralized manner in the network of interest. Based on the results of those experiments, we compare Relational A2C to other reinforcement learning algorithms, like Q-Routing and Hybrid Routing. This comparison illustrates that solving the joint optimization problem increases network efficiency and reduces selfish agent behavior.

Multiagent Meta-Reinforcement Learning for Adaptive Multipath Routing Optimization

Toward Packet Routing With Fully Distributed Multiagent Deep Reinforcement Learning

Toward Packet Routing with Fully-distributed Multi-agent Deep Reinforcement Learning

Multi-agent reinforcement learning for network routing in integrated access backhaul networks

Packet Routing with Graph Attention Multi-agent Reinforcement Learning

Routing optimization with Monte Carlo Tree Search-based multi-agent reinforcement learning

A Multi-agent Reinforcement Learning Routing Protocol in Mobile Robot Network

Scalable QoS-Aware Multipath Routing in Hybrid Knowledge-Defined Networking with Multi-Agent Deep Reinforcement Learning

Multi-agent Deep Reinforcement Learning for Resilience-Driven Routing and Scheduling of Mobile Energy Storage Systems

A Multi-Policy Deep Reinforcement Learning Approach for Multi-Objective Joint Routing and Scheduling in Deterministic Networks

Distributed Routing Optimization Algorithm for FANET Based on Multi-agent Reinforcement Learning

Distributed Traffic Engineering in Hybrid Software Defined Networks: A Multi-agent Reinforcement Learning Framework

Distributed Multi-Hop Traffic Engineering Via Stochastic Policy Gradient Reinforcement Learning

A Multi-Agent Reinforcement Learning Routing Protocol for Underwater Optical Sensor Networks

Packet Routing Against Network Congestion: A Deep Multi-agent Reinforcement Learning Approach

Traffic-driven Spectrum and Power Allocation Via Scalable Multi-Agent Reinforcement Learning

Multi-Vehicle Routing Problems with Soft Time Windows: A Multi-Agent Reinforcement Learning Approach

Adaptive and Robust Routing with Lyapunov-Based Deep RL in MEC Networks Enabled by Blockchains.

Multi-Agent Path Finding via Reinforcement Learning with Hybrid Reward

RTHop: Real-Time Hop-by-Hop Mobile Network Routing by Decentralized Learning With Semantic Attention

A Meta Multi-agent Reinforcement Learning Algorithm for Multi-intersection Traffic Signal Control