Abstract:Software-Defined Network (SDN) paradigm decouples control plane from data plane and provides a logically-centralized control to whole underlying network, which enables the controller to centrally configure the network, and has demonstrated great potential in fully utilizing network capacity. However, the traditional SDN does not consider the accumulative long-term revenue in routing configuration for network service provider (SP). For example, it is prone to choosing the path passing through important nodes, which may increase the possibility of future congestion, and hamper the SP’s long-term revenue. Deep reinforcement learning (DRL) has brought about the transformative advances in decision-making when facing uncertainty. As a well-known type of DRL, Deep Q-network (DQN) can explore optimal routing schemes from high dimensional network state. However, most of existing DRL based methods fail to provide satisfactory configuration when confronted with a topology unseen during training. Meanwhile, many existing network optimization schemes ignore the impact of nodes and global network attributes on the routing configuration. Consequently, they may perform poorly in real scenarios. In this paper, we propose a graph reinforcement learning (GRL) based SDN routing selection scheme, GN-DQN for optimizing SP’s accumulative long-term revenue. First, besides the link embeddings, GN-DQN explicitly learns the representations of the routers and globe network topology with full graph neural network block. Then, through utilizing the formed network representation as the state, DQN is used to learn to sequentially choose the optimal routing path from the candidate path set to maximize the accumulative long-term revenue. Thorough experiments on multiple real network topologies show that our proposed GN-DQN outperforms other GRL based and traditional routing selection schemes, and can generalize over arbitrary topologies. Moreover, GN-DQN also demonstrates great robustness to the link failure scenario.

Combining graph neural network with deep reinforcement learning for resource allocation in computing force networks

Adaptive Routing of Task in Computing Force Network by Integrating Graph Convolutional Network and Deep Q-Network.

Combining Deep Reinforcement Learning with Graph Neural Networks for Optimal VNF Placement

Intelligent Joint Network Slicing and Routing via GCN-powered Multi-Task Deep Reinforcement Learning

An Approach to Combine the Power of Deep Reinforcement Learning with a Graph Neural Network for Routing Optimization

Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use case

Joint Routing and Scheduling Optimization in Time-Sensitive Networks Using Graph-Convolutional-Network-Based Deep Reinforcement Learning.

Offloading DNN Tasks Based on Graph Reinforcement Learning in Client-Edge-Cloud Environment

GNN-Based Hierarchical Deep Reinforcement Learning for NFV-Oriented Online Resource Orchestration in Elastic Optical DCIs

GROM: A Generalized Routing Optimization Method with Graph Neural Network and Deep Reinforcement Learning

Network Aware Compute and Memory Allocation in Optically Composable Data Centres with Deep Reinforcement Learning and Graph Neural Networks

A Graph Convolutional Network-Based Deep Reinforcement Learning Approach for Resource Allocation in a Cognitive Radio Network

G-Routing: Graph Neural Networks-Based Flexible Online Routing

Task Scheduling for ICN-Based Computing First Network: A Deep Reinforcement Learning Approach

Graph Attention Network-Based Multi-Agent Reinforcement Learning for Slicing Resource Management in Dense Cellular Network

Graph Reinforcement Learning for Radio Resource Allocation

GCNPart: Interference-Aware Resource Partitioning Framework with Graph Convolutional Neural Networks and Deep Reinforcement Learning.

Towards an energy-efficient Data Center Network based on deep reinforcement learning

Graph Neural Networks and Deep Reinforcement Learning Based Resource Allocation for V2X Communications

A Graph reinforcement learning based SDN routing path selection for optimizing long-term revenue

Hierarchical Collaborative Resource Scheduling in Industrial Internet of Things Based on Graph Neural Networks and Deep Reinforcement Learning