Abstract:Traditional routing typically relies on simpler performance metrics that can be derived directly through mathematical methods for decision-making, which often results in limited optimization outcomes. As future networks expand, along with the diversity of applications and traffic volume, the network environment grows increasingly complex. In contrast, Intelligent Routing (IR) that leverages machine learning methods can model more complex performance metrics, rendering it better suited to the intricate scenarios of future networks. The increasing complexity of networks also indicates that the workload associated with collecting routing information and executing decision calculations is growing exponentially. Compared to centralized IR, Distributed IR (DIR) distributes the computational load and interaction demands across multiple nodes, thereby offering enhanced scalability. However, DIR makes decisions based on local information, which limits global optimization. In this paper, we propose a novel Scalable Intelligent Routing based on Distributed Graph Reinforcement Learning, called ScaIR. ScaIR is a full y distributed multi-agent routing method. Each router is an independent agent based on local graph Reinforcement Learning (RL). Graph Neural Networks (GNN) are employed to extract global network characteristics which serve as input for RL, thereby enhancing global optimization. Especially, GNN here is also fully distributed. Each router has an independent sub-GNN determined by the adjacency relationships with its one-hop neighbors. Instead of entire network information and model parameters, each sub-GNN only iteratively interacts with its neighbors and computes a highly compressed Feature Vector (FV) representing the current network state, which greatly saves the computing and communication cost. We carried out extensive simulation experiments under multiple real network topologies of different scales. Simulation results show that ScaIR reduces forwarding time by more than 25% and achieves faster convergence. It can better adapt to congested, dynamic or unknown environments. Compared to other methods, it significantly reduces communication cost and computational time, and has better scalability. In addition, by changing the FV length of sub-GNNs, it is verified that GNN does play a key role in ScaIR.

SINET - Enabling Scalable Network Routing with Deep Reinforcement Learning on Partial Nodes.

Improving the Scalability of Deep Reinforcement Learning-Based Routing with Control on Partial Nodes

Enabling Scalable Routing in Software-Defined Networks With Deep Reinforcement Learning on Critical Nodes

An Intelligent Routing Mechanism Based on Deep Reinforcement Learning

A Hybrid Deep Reinforcement Learning Routing Method under Dynamic and Complex Traffic with Software Defined Networking

Towards Cognitive Routing based on Deep Reinforcement Learning

An Intelligent SDWN Routing Algorithm Based on Network Situational Awareness and Deep Reinforcement Learning

A Routing Optimization Method for Software-Defined SGIN Based on Deep Reinforcement Learning

Deep-Reinforcement-Learning-Based Intelligent Routing Strategy for FANETs

Deep Graph Reinforcement Learning Based Intelligent Traffic Routing Control for Software-Defined Wireless Sensor Networks

Enabling efficient routing for traffic engineering in SDN with Deep Reinforcement Learning

A Graph reinforcement learning based SDN routing path selection for optimizing long-term revenue

Scalable Deep Reinforcement Learning-Based Online Routing for Multi-Type Service Requirements

ScaIR: Scalable Intelligent Routing based on Distributed Graph Reinforcement Learning

QoS Routing Optimization Based on Deep Reinforcement Learning in SDN

Scalable Deep Reinforcement Learning for Routing and Spectrum Access in Physical Layer

A Deep Reinforcement Learning Approach for Adaptive Traffic Routing in Next-gen Networks

Routing Optimization With Deep Reinforcement Learning in Knowledge Defined Networking

GTD3-NET: A deep reinforcement learning-based routing optimization algorithm for wireless networks

Intelligent Routing Algorithm over SDN: Reusable Reinforcement Learning Approach

TITE: A Transformer-Based Deep Reinforcement Learning Approach for Traffic Engineering in Hybrid SDN with Dynamic Traffic