Abstract:Routing optimization is a critical issue to Internet Service Providers (ISPs) since it is highly related to the network traffic management efficiency. State-of-the-art approaches utilize Reinforcement Learning (RL) technology to minimize Maximum Link Utilization (MLU). However, these approaches have two significant shortcomings: 1) the rudimentary structure of Neural Networks (NNs) restricts their ability to capture network traffic patterns accurately, resulting in sub-optimal critical flow selection, and 2) the training algorithm is ineffective in utilizing experience, leading to training instability and prolonged duration. To overcome the above shortcomings, we propose ERODRL, a network traffic management scheme which incorporates a RL based critical flow selection strategy and a flow rerouting algorithm, towards the goal of MLU minimization. In details, the RL based critical flow selection strategy incorporates a Convolutional Neural Network (CNN) block to analyze traffic features and utilizes the GELU activation function to improve traffic distribution matching. It then introduces full connection layers to the actor and critic networks respectively, allowing for detailed traffic behavior analysis. To improve training efficiency, a trust region algorithm is also involved fast the training speed of RL agent. After the critical flow are identified, the flow routing problem is formulated into Linear Programming (LP) form and a low-complexity rerouting solution is proposed to better fit the traffic patterns of ISPs. Extensive simulation results are conducted and the results demonstrate that ERODRL improves training speeds by 2.3 to 8.7 times compared to previous RL-based schemes, and reduces the MLU by 43.24% and 16.01% averagely compared to state-of-the-art solutions.

On Deep Reinforcement Learning for Static Routing and Wavelength Assignment

A Multi-Task-Learning-Based Transfer Deep Reinforcement Learning Design for Autonomic Optical Networks

Opti-DeepRoute: A Topology-Adaptive Deep Reinforcement Learning Based Service Provisioning Framework for Elastic Optical Network

Towards Real-Time Routing Optimization with Deep Reinforcement Learning: Open Challenges

A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning

Routing Optimization With Deep Reinforcement Learning in Knowledge Defined Networking

Self-learning Routing for Optical Networks.

Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use case

Techniques for applying reinforcement learning to routing and wavelength assignment problems in optical fiber communication networks

Towards Cognitive Routing based on Deep Reinforcement Learning

ERODRL: Efficient ISP Routing Optimization Via Deep Reinforcement Learning

Deep learning-optical network routing algorithm based on wavelength continuity supervision

Deep Reinforcement Learning for Router Selection in Network with Heavy Traffic

A Deep Reinforcement Learning Approach for Adaptive Traffic Routing in Next-gen Networks

Relational Deep Reinforcement Learning for Routing in Wireless Networks

DRLAR: A Deep Reinforcement Learning-Based Adaptive Routing Framework for Network-on-chips

Enabling Scalable Routing in Software-Defined Networks With Deep Reinforcement Learning on Critical Nodes

Heuristic Reward Design for Deep Reinforcement Learning-Based Routing, Modulation and Spectrum Assignment of Elastic Optical Networks

DeepRMSA: A Deep Reinforcement Learning Framework for Routing, Modulation and Spectrum Assignment in Elastic Optical Networks

OpticGAI: Generative AI-aided Deep Reinforcement Learning for Optical Networks Optimization

Scalable Deep Reinforcement Learning-Based Online Routing for Multi-Type Service Requirements