Abstract:Software-Defined Network (SDN) paradigm decouples control plane from data plane and provides a logically-centralized control to whole underlying network, which enables the controller to centrally configure the network, and has demonstrated great potential in fully utilizing network capacity. However, the traditional SDN does not consider the accumulative long-term revenue in routing configuration for network service provider (SP). For example, it is prone to choosing the path passing through important nodes, which may increase the possibility of future congestion, and hamper the SP’s long-term revenue. Deep reinforcement learning (DRL) has brought about the transformative advances in decision-making when facing uncertainty. As a well-known type of DRL, Deep Q-network (DQN) can explore optimal routing schemes from high dimensional network state. However, most of existing DRL based methods fail to provide satisfactory configuration when confronted with a topology unseen during training. Meanwhile, many existing network optimization schemes ignore the impact of nodes and global network attributes on the routing configuration. Consequently, they may perform poorly in real scenarios. In this paper, we propose a graph reinforcement learning (GRL) based SDN routing selection scheme, GN-DQN for optimizing SP’s accumulative long-term revenue. First, besides the link embeddings, GN-DQN explicitly learns the representations of the routers and globe network topology with full graph neural network block. Then, through utilizing the formed network representation as the state, DQN is used to learn to sequentially choose the optimal routing path from the candidate path set to maximize the accumulative long-term revenue. Thorough experiments on multiple real network topologies show that our proposed GN-DQN outperforms other GRL based and traditional routing selection schemes, and can generalize over arbitrary topologies. Moreover, GN-DQN also demonstrates great robustness to the link failure scenario.

Decentralized Routing and Radio Resource Allocation in Wireless Ad Hoc Networks via Graph Reinforcement Learning

Graph Reinforcement Learning for Radio Resource Allocation

G-Routing: Graph Neural Networks-Based Flexible Online Routing

GTD3-NET: A deep reinforcement learning-based routing optimization algorithm for wireless networks

Toward Packet Routing With Fully Distributed Multiagent Deep Reinforcement Learning

Toward Packet Routing with Fully-distributed Multi-agent Deep Reinforcement Learning

Scalable Deep Reinforcement Learning for Routing and Spectrum Access in Physical Layer

Packet Routing with Graph Attention Multi-agent Reinforcement Learning

Deep-Reinforcement-Learning-Based Intelligent Routing Strategy for FANETs

Load-Aware Distributed Resource Allocation for MF-TDMA Ad Hoc Networks: A Multi-Agent DRL Approach.

Deep Graph Reinforcement Learning Based Intelligent Traffic Routing Control for Software-Defined Wireless Sensor Networks

A Graph reinforcement learning based SDN routing path selection for optimizing long-term revenue

An Intelligent SDWN Routing Algorithm Based on Network Situational Awareness and Deep Reinforcement Learning

Routing Optimization With Deep Reinforcement Learning in Knowledge Defined Networking

Reinforcement Learning based Energy-Efficient Fast Routing for FANETs

Deep Learning based Wireless Resource Allocation with Application to Vehicular Networks

Multi-agent Deep Reinforcement Learning for Cross-Layer Scheduling in Mobile Ad-Hoc Networks

Multi-Agent Reinforcement Learning for Power Control in Wireless Networks via Adaptive Graphs

Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use case

An adaptive intelligent routing algorithm based on deep reinforcement learning

A Deep Reinforcement Learning Approach for Adaptive Traffic Routing in Next-gen Networks