Abstract:Deterministic Networking (DetNet) is a highly predictable and controllable network technology. It provides low packet loss rate and bounded latency data transmission for applications through resource reservation and scheduling mechanisms. However, DetNet is a hybrid traffic system, and the resource reservation mechanism cannot guarantee the deterministic requirements as the number of diverse deterministic applications increases. As a result, there is an urgent need for an efficient and fine-grained scheduling mechanism to meet the deterministic and bounded latency requirements. In this paper, we propose a novel end-to-end multi-policy deep reinforcement learning framework for automatically learning multiple policies and addressing the problem of multi-objective joint routing and scheduling. Specifically, we formulate the multi-action problem in joint routing and scheduling as a Multi-Markov Decision Process (MMDP) and design a new reward function to optimize multiple objectives. When optimizing the learning agent, we introduce an A3C-based multi-strategy optimization algorithm (A3C-MSO) to train two sub-policies, including the queue operation policy and the node operation policy for assigning queue operations to nodes. Furthermore, we integrate a graph convolutional network (GCN) into the learning framework to capture the spatial characteristics of irregular network topologies and enhance the algorithm’s generalization ability. Extensive experimental results in different scenarios indicate that compared to the existing state-of-the-art mechanisms, the proposed mechanism has shown a 13% improvement in schedulability and an 18% enhancement in resource utilization. Particularly in high-load scenarios, the time cost of the proposed mechanism can be reduced by up to 40.5%. Furthermore, results obtained on real industrial network topology instances indicate that the proposed learning strategies exhibit good generalization and effectiveness in large-scale scheduling instances.

Graph-neural-network-based delay estimation for communication networks with heterogeneous scheduling policies

RouteNet-Fermi: Network Modeling with Graph Neural Networks

Delay-Oriented Distributed Scheduling Using Graph Neural Networks

Railway Network Delay Evolution: A Heterogeneous Graph Neural Network Approach

Graph Neural Modeling of Network Flows

G-Routing: Graph Neural Networks-Based Flexible Online Routing

Wireless D2D Network Link Scheduling Based on Graph Embedding

Wireless Network Scheduling with Discrete Propagation Delays: Theorems and Algorithms

Delay-aware Backpressure Routing Using Graph Neural Networks

An Approach to Combine the Power of Deep Reinforcement Learning with a Graph Neural Network for Routing Optimization

Xnet: Modeling Network Performance with Graph Neural Networks

Throughput Optimal Decentralized Scheduling of Multi-Hop Networks with End-to-End Deadline Constraints: II Wireless Networks with Interference

Deeper and Wider Networks for Performance Metrics Prediction in Communication Networks

Packet Routing with Graph Attention Multi-agent Reinforcement Learning

Graph neural network-based scheduling for multi-UAV-enabled communications in D2D networks

Investigation of a GNN approach to mitigate congestion in a realistic MANET scenario

A Multi-Policy Deep Reinforcement Learning Approach for Multi-Objective Joint Routing and Scheduling in Deterministic Networks

Open World Learning Graph Convolution for Latency Estimation in Routing Networks

Graph Neural Networks for Routing Optimization: Challenges and Opportunities

Xnet: Improving Expressiveness and Granularity for Network Modeling with Graph Neural Networks

Distributed Link Sparsification for Scalable Scheduling Using Graph Neural Networks