Abstract:Software-Defined Network (SDN) paradigm decouples control plane from data plane and provides a logically-centralized control to whole underlying network, which enables the controller to centrally configure the network, and has demonstrated great potential in fully utilizing network capacity. However, the traditional SDN does not consider the accumulative long-term revenue in routing configuration for network service provider (SP). For example, it is prone to choosing the path passing through important nodes, which may increase the possibility of future congestion, and hamper the SP’s long-term revenue. Deep reinforcement learning (DRL) has brought about the transformative advances in decision-making when facing uncertainty. As a well-known type of DRL, Deep Q-network (DQN) can explore optimal routing schemes from high dimensional network state. However, most of existing DRL based methods fail to provide satisfactory configuration when confronted with a topology unseen during training. Meanwhile, many existing network optimization schemes ignore the impact of nodes and global network attributes on the routing configuration. Consequently, they may perform poorly in real scenarios. In this paper, we propose a graph reinforcement learning (GRL) based SDN routing selection scheme, GN-DQN for optimizing SP’s accumulative long-term revenue. First, besides the link embeddings, GN-DQN explicitly learns the representations of the routers and globe network topology with full graph neural network block. Then, through utilizing the formed network representation as the state, DQN is used to learn to sequentially choose the optimal routing path from the candidate path set to maximize the accumulative long-term revenue. Thorough experiments on multiple real network topologies show that our proposed GN-DQN outperforms other GRL based and traditional routing selection schemes, and can generalize over arbitrary topologies. Moreover, GN-DQN also demonstrates great robustness to the link failure scenario.

Deep Reinforcement Learning-Based Routing and Spectrum Assignment of EONs by Exploiting GCN and RNN for Feature Extraction

DeepRMSA: A Deep Reinforcement Learning Framework for Routing, Modulation and Spectrum Assignment in Elastic Optical Networks

Deep-RMSA: A Deep-Reinforcement-Learning Routing, Modulation and Spectrum Assignment Agent for Elastic Optical Networks

Reconfiguring Multicast Sessions in Elastic Optical Networks Adaptively with Graph-Aware Deep Reinforcement Learning

On Deep Reinforcement Learning for Static Routing and Wavelength Assignment

A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning

GNN-Based Hierarchical Deep Reinforcement Learning for NFV-Oriented Online Resource Orchestration in Elastic Optical DCIs

Multi-Agent and Cooperative Deep Reinforcement Learning for Scalable Network Automation in Multi-Domain SD-EONs.

Graph-Model-Based Dynamic Routing and Spectrum Assignment in Elastic Optical Networks

Resource Allocation in Multicore Elastic Optical Networks: A Deep Reinforcement Learning Approach

Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use case

Enhancing Routing in SD-EONs through Reinforcement Learning: A Comparative Analysis

Reconfiguring Multicast Sessions in EONs Adaptively with Deep Reinforcement Learning: (Invited Paper)

Enabling High-Throughput Routing for LEO Satellite Broadband Networks: A Flow-Centric Deep Reinforcement Learning Approach

Routing, Modulation Level, Spectrum and Transceiver Assignment in Elastic Optical Networks.

A Multi-Task-Learning-Based Transfer Deep Reinforcement Learning Design for Autonomic Optical Networks

GRouting: Dynamic Routing for LEO Satellite Networks with Graph-based Deep Reinforcement Learning

Time-Efficient RSA over Large-Scale Multi-Domain EON

A Graph reinforcement learning based SDN routing path selection for optimizing long-term revenue

A Survey of Intelligent End-to-End Networking Solutions: Integrating Graph Neural Networks and Deep Reinforcement Learning Approaches

Towards Cognitive Routing based on Deep Reinforcement Learning