Abstract:Deterministic Networking (DetNet) is a highly predictable and controllable network technology. It provides low packet loss rate and bounded latency data transmission for applications through resource reservation and scheduling mechanisms. However, DetNet is a hybrid traffic system, and the resource reservation mechanism cannot guarantee the deterministic requirements as the number of diverse deterministic applications increases. As a result, there is an urgent need for an efficient and fine-grained scheduling mechanism to meet the deterministic and bounded latency requirements. In this paper, we propose a novel end-to-end multi-policy deep reinforcement learning framework for automatically learning multiple policies and addressing the problem of multi-objective joint routing and scheduling. Specifically, we formulate the multi-action problem in joint routing and scheduling as a Multi-Markov Decision Process (MMDP) and design a new reward function to optimize multiple objectives. When optimizing the learning agent, we introduce an A3C-based multi-strategy optimization algorithm (A3C-MSO) to train two sub-policies, including the queue operation policy and the node operation policy for assigning queue operations to nodes. Furthermore, we integrate a graph convolutional network (GCN) into the learning framework to capture the spatial characteristics of irregular network topologies and enhance the algorithm’s generalization ability. Extensive experimental results in different scenarios indicate that compared to the existing state-of-the-art mechanisms, the proposed mechanism has shown a 13% improvement in schedulability and an 18% enhancement in resource utilization. Particularly in high-load scenarios, the time cost of the proposed mechanism can be reduced by up to 40.5%. Furthermore, results obtained on real industrial network topology instances indicate that the proposed learning strategies exhibit good generalization and effectiveness in large-scale scheduling instances.

DGS: an Efficient Delay-Guaranteed Scheduling Framework for Wireless Deterministic Networking

Joint optimization of time-slot allocation and traffic steering for large-scale deterministic networks

Time Sensitive Data Access for Massive Users: A Mean-Field Approximation Approach

A New Packet Scheduling Algorithm in Wireless Cdma Data Networks

Integrating Deterministic Networking with 5G

Interference Coordination in Full-Duplex HetNet with Large-Scale Antenna Arrays.

A Multi-Policy Deep Reinforcement Learning Approach for Multi-Objective Joint Routing and Scheduling in Deterministic Networks

Deterministic Latency Bounded Network Slice Deployment in IP-Over-WDM Based Metro-Aggregation Networks

A Scalable Per-flow Priority Scheduling Scheme for High-Speed Network

Congestion Control with Deterministic Service Delay Guarantee

DetCNCS: Deterministic Computing and Networking Convergence Scheduling.

Deep Reinforcement Learning-Based Joint Scheduling of 5G and TSN in Industrial Networks

Long-distance Deterministic Transmission among TSN Networks: Converging CQF and DIP

A Simulation Framework for Contention-Free Scheduling on WiGig

Ultra-Low Latency Wireless Communications for Deterministic Networking: A Cross-Layer Approach

Delay-Oriented Distributed Scheduling Using Graph Neural Networks

Meeting Hard Delay Constraint in Massive Access: A Mean-field Approach

Distributed Scheduling for Throughput Maximization under Deadline Constraint in Wireless Mesh Networks

Multi-Agent DRL-Based Deadline-Aware Routing for Deterministic Wide-Area Networks

DetSpace: Distributed Congestion Free Routing for Global Deterministic Network in Mega-Constellations

An Intelligent Deterministic Scheduling Method for Ultralow Latency Communication in Edge Enabled Industrial Internet of Things