Abstract:Emerging applications raise critical QoS requirements for the Internet. The improvements in flow classification technologies, software-defined networks (SDN), and programmable network devices make it possible to fast identify users’ requirements and control the routing for fine-grained traffic flows. Meanwhile, the problem of optimizing the forwarding paths for traffic flows with multiple QoS requirements in an online fashion is not addressed sufficiently. To address the problem, we propose DRL-OR-S, a highly scalable online routing algorithm using multi-agent deep reinforcement learning. DRL-OR-S adopts a comprehensive reward function, an efficient learning algorithm, and a novel deep neural network structure to learn appropriate routing strategies for different types of flow requirements. In order to enhance the generalization and scalability, we propose a novel graph-based actor-critic network architecture and a carefully designed input state for DRL-OR-S. To accelerate the training process and guarantee reliability, we further introduce an NN-simulator for efficient offline training and a safe learning mechanism to avoid unsafe routes during the online routing process. We implement DRL-OR-S under SDN architecture and conduct Mininet-based experiments using real network topologies and traffic traces. The results validate that DRL-OR-S can well satisfy the requirements of latency-sensitive, throughput-sensitive, latency-throughput-sensitive, and latency-loss-sensitive flows at the same time, while exhibiting great adaptiveness and reliability under the scenarios of link failure, traffic change, unseen large topology and partial deployment.

Towards Real-Time Routing Optimization with Deep Reinforcement Learning: Open Challenges

Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use case

On Deep Reinforcement Learning for Static Routing and Wavelength Assignment

Deep Reinforcement Learning for Router Selection in Network with Heavy Traffic

Routing Optimization With Deep Reinforcement Learning in Knowledge Defined Networking

A Deep Reinforcement Learning Approach for Adaptive Traffic Routing in Next-gen Networks

Towards Cognitive Routing based on Deep Reinforcement Learning

A Hybrid Deep Reinforcement Learning Routing Method under Dynamic and Complex Traffic with Software Defined Networking

Improving the Scalability of Deep Reinforcement Learning-Based Routing with Control on Partial Nodes

DQS: A QoS-Driven Routing Optimization Approach in SDN using Deep Reinforcement Learning

GTD3-NET: A deep reinforcement learning-based routing optimization algorithm for wireless networks

Leveraging Deep Reinforcement Learning for Traffic Engineering: A Survey

G-Routing: Graph Neural Networks-Based Flexible Online Routing

Scalable Deep Reinforcement Learning-Based Online Routing for Multi-Type Service Requirements

A Deep-Reinforcement Learning Approach for Software-Defined Networking Routing Optimization

Toward Packet Routing With Fully Distributed Multiagent Deep Reinforcement Learning

Enabling Scalable Routing in Software-Defined Networks With Deep Reinforcement Learning on Critical Nodes

Toward Packet Routing with Fully-distributed Multi-agent Deep Reinforcement Learning

Distributed and Adaptive Traffic Engineering with Deep Reinforcement Learning

Tensor-Based Reinforcement Learning for Network Routing

On Design and Implementation of Reinforcement Learning Based Cognitive Routing for Autonomous Networks