Abstract:Deterministic Networking (DetNet) is a highly predictable and controllable network technology. It provides low packet loss rate and bounded latency data transmission for applications through resource reservation and scheduling mechanisms. However, DetNet is a hybrid traffic system, and the resource reservation mechanism cannot guarantee the deterministic requirements as the number of diverse deterministic applications increases. As a result, there is an urgent need for an efficient and fine-grained scheduling mechanism to meet the deterministic and bounded latency requirements. In this paper, we propose a novel end-to-end multi-policy deep reinforcement learning framework for automatically learning multiple policies and addressing the problem of multi-objective joint routing and scheduling. Specifically, we formulate the multi-action problem in joint routing and scheduling as a Multi-Markov Decision Process (MMDP) and design a new reward function to optimize multiple objectives. When optimizing the learning agent, we introduce an A3C-based multi-strategy optimization algorithm (A3C-MSO) to train two sub-policies, including the queue operation policy and the node operation policy for assigning queue operations to nodes. Furthermore, we integrate a graph convolutional network (GCN) into the learning framework to capture the spatial characteristics of irregular network topologies and enhance the algorithm’s generalization ability. Extensive experimental results in different scenarios indicate that compared to the existing state-of-the-art mechanisms, the proposed mechanism has shown a 13% improvement in schedulability and an 18% enhancement in resource utilization. Particularly in high-load scenarios, the time cost of the proposed mechanism can be reduced by up to 40.5%. Furthermore, results obtained on real industrial network topology instances indicate that the proposed learning strategies exhibit good generalization and effectiveness in large-scale scheduling instances.

Deterministic Networking (DetNet) Configuration YANG Model

Deterministic Networking Flow Information Model

SRv6 for Deterministic Networking (DetNet)

Agent-based En-Route Diversion: Dynamic Behavioral Responses and Network Performance Represented by Macroscopic Fundamental Diagrams

Evolving Defense Mechanism for Future Network Security

Engineering Traffic Uncertainty In The Openflow Data Plane

Deterministic Computing Power Networking: Architecture, Technologies and Prospects

Large-Scale Deterministic Networks: Architecture, Enabling Technologies, Case Study and Future Directions

DGS: an Efficient Delay-Guaranteed Scheduling Framework for Wireless Deterministic Networking

Integrating Deterministic Networking with 5G

A Programmable Network Management Architecture for Address Driven Network

Deterministic Latency Bounded Network Slice Deployment in IP-Over-WDM Based Metro-Aggregation Networks

DetSpace: Distributed Congestion Free Routing for Global Deterministic Network in Mega-Constellations

Deterministic Service Function Chaining over Beyond 5G Edge Fabric

Identifying Hybrid DDoS Attacks in Deterministic Machine-to-Machine Networks on a Per-Deterministic-Flow Basis

DetCNCS: Deterministic Computing and Networking Convergence Scheduling.

Xnet: Modeling Network Performance with Graph Neural Networks

A Multi-Policy Deep Reinforcement Learning Approach for Multi-Objective Joint Routing and Scheduling in Deterministic Networks

A Framework for Management and Control of Microwave and Millimeter Wave Interface Parameters - Problem Statement

Ultra-Low Latency Wireless Communications for Deterministic Networking: A Cross-Layer Approach

A YANG Data Model for Microwave Topology