Abstract:Deterministic Networking (DetNet) is a highly predictable and controllable network technology. It provides low packet loss rate and bounded latency data transmission for applications through resource reservation and scheduling mechanisms. However, DetNet is a hybrid traffic system, and the resource reservation mechanism cannot guarantee the deterministic requirements as the number of diverse deterministic applications increases. As a result, there is an urgent need for an efficient and fine-grained scheduling mechanism to meet the deterministic and bounded latency requirements. In this paper, we propose a novel end-to-end multi-policy deep reinforcement learning framework for automatically learning multiple policies and addressing the problem of multi-objective joint routing and scheduling. Specifically, we formulate the multi-action problem in joint routing and scheduling as a Multi-Markov Decision Process (MMDP) and design a new reward function to optimize multiple objectives. When optimizing the learning agent, we introduce an A3C-based multi-strategy optimization algorithm (A3C-MSO) to train two sub-policies, including the queue operation policy and the node operation policy for assigning queue operations to nodes. Furthermore, we integrate a graph convolutional network (GCN) into the learning framework to capture the spatial characteristics of irregular network topologies and enhance the algorithm’s generalization ability. Extensive experimental results in different scenarios indicate that compared to the existing state-of-the-art mechanisms, the proposed mechanism has shown a 13% improvement in schedulability and an 18% enhancement in resource utilization. Particularly in high-load scenarios, the time cost of the proposed mechanism can be reduced by up to 40.5%. Furthermore, results obtained on real industrial network topology instances indicate that the proposed learning strategies exhibit good generalization and effectiveness in large-scale scheduling instances.

Multi-Agent Reinforcement Learning for Wireless User Scheduling: Performance, Scalablility, and Generalization

Multi-Agent Reinforcement Learning for Multi-Cell Spectrum and Power Allocation

Traffic-driven Spectrum and Power Allocation Via Scalable Multi-Agent Reinforcement Learning

Scalable Multi-Agent Reinforcement Learning for Residential Load Scheduling under Data Governance

Scalable Multi-agent Reinforcement Learning for Factory-wide Dynamic Scheduling

Exploring Multi-Agent Reinforcement Learning for Unrelated Parallel Machine Scheduling

Distributed Adaptive Multiuser Scheduling Via Multi-Agent Reinforcement Learning in Multicell MIMO Cellular Networks

Multi-Agent Reinforcement Learning for Real-Time Dynamic Production Scheduling in a Robot Assembly Cell

Scalable Joint Learning of Wireless Multiple-Access Policies and their Signaling

Efficient Communications in Multi-Agent Reinforcement Learning for Mobile Applications

Multi-agent Deep Reinforcement Learning for Cross-Layer Scheduling in Mobile Ad-Hoc Networks

Multi-Agent Reinforcement Learning Approach for Residential Microgrid Energy Scheduling

Multi-Agent Reinforcement Learning for Power System Operation and Control

Multi-agent Reinforcement Learning for Dynamic Dispatching in Material Handling Systems

Lyapunov-guided Multi-Agent Reinforcement Learning for Delay-Sensitive Wireless Scheduling

Multi-agent hierarchical reinforcement learning for energy management

Multistep Multiagent Reinforcement Learning for Optimal Energy Schedule Strategy of Charging Stations in Smart Grid.

Multi-Agent Reinforcement Learning for Network Load Balancing in Data Center

Deep Reinforcement Learning for Wireless Scheduling in Distributed Networked Control

Decentralized Multi-agent Reinforcement Learning with Multi-time Scale of Decision Epochs

A Multi-Policy Deep Reinforcement Learning Approach for Multi-Objective Joint Routing and Scheduling in Deterministic Networks