Abstract:Software-Defined Network (SDN) paradigm decouples control plane from data plane and provides a logically-centralized control to whole underlying network, which enables the controller to centrally configure the network, and has demonstrated great potential in fully utilizing network capacity. However, the traditional SDN does not consider the accumulative long-term revenue in routing configuration for network service provider (SP). For example, it is prone to choosing the path passing through important nodes, which may increase the possibility of future congestion, and hamper the SP’s long-term revenue. Deep reinforcement learning (DRL) has brought about the transformative advances in decision-making when facing uncertainty. As a well-known type of DRL, Deep Q-network (DQN) can explore optimal routing schemes from high dimensional network state. However, most of existing DRL based methods fail to provide satisfactory configuration when confronted with a topology unseen during training. Meanwhile, many existing network optimization schemes ignore the impact of nodes and global network attributes on the routing configuration. Consequently, they may perform poorly in real scenarios. In this paper, we propose a graph reinforcement learning (GRL) based SDN routing selection scheme, GN-DQN for optimizing SP’s accumulative long-term revenue. First, besides the link embeddings, GN-DQN explicitly learns the representations of the routers and globe network topology with full graph neural network block. Then, through utilizing the formed network representation as the state, DQN is used to learn to sequentially choose the optimal routing path from the candidate path set to maximize the accumulative long-term revenue. Thorough experiments on multiple real network topologies show that our proposed GN-DQN outperforms other GRL based and traditional routing selection schemes, and can generalize over arbitrary topologies. Moreover, GN-DQN also demonstrates great robustness to the link failure scenario.

Improved Exploration Strategy for Q-Learning Based Multipath Routing in SDN Networks

A Multipath Resource Updating Approach for Distributed Controllers in Software-Defined Network

DQS: A QoS-Driven Routing Optimization Approach in SDN using Deep Reinforcement Learning

Application-aware QoS routing in SDNs using machine learning techniques

Enhancing Routing in SD-EONs through Reinforcement Learning: A Comparative Analysis

Multi-Path Routing Algorithm Based on Deep Reinforcement Learning for SDN

Intelligent Routing Algorithm over SDN: Reusable Reinforcement Learning Approach

AQROM: A quality of service aware routing optimization mechanism based on asynchronous advantage actor-critic in software-defined networks

AQMDRL: Automatic Quality of Service Architecture Based on Multistep Deep Reinforcement Learning in Software-Defined Networking

An Intelligent SDWN Routing Algorithm Based on Network Situational Awareness and Deep Reinforcement Learning

Research on Routing Optimization of SDN Network Using Reinforcement Learning Method

A Routing Optimization Method for Software-Defined Optical Transport Networks Based on Ensembles and Reinforcement Learning

Routing Optimization With Deep Reinforcement Learning in Knowledge Defined Networking

SDN Control Strategy and QoS Optimization Simulation Performance Based on Improved Algorithm

An Intelligent Routing Algorithm Based on Prioritized Replay Double DQN for MANET

A Hybrid Deep Reinforcement Learning Routing Method under Dynamic and Complex Traffic with Software Defined Networking

Packet Routing Against Network Congestion: A Deep Multi-agent Reinforcement Learning Approach

Reinforcement learning-based SDN routing scheme empowered by causality detection and GNN

A Graph reinforcement learning based SDN routing path selection for optimizing long-term revenue

Reinforcement learning based dynamic distributed routing scheme for mega LEO satellite networks

Traffic Engineering in Hybrid Software Defined Network Via Reinforcement Learning