Abstract:Deterministic Networking (DetNet) is a highly predictable and controllable network technology. It provides low packet loss rate and bounded latency data transmission for applications through resource reservation and scheduling mechanisms. However, DetNet is a hybrid traffic system, and the resource reservation mechanism cannot guarantee the deterministic requirements as the number of diverse deterministic applications increases. As a result, there is an urgent need for an efficient and fine-grained scheduling mechanism to meet the deterministic and bounded latency requirements. In this paper, we propose a novel end-to-end multi-policy deep reinforcement learning framework for automatically learning multiple policies and addressing the problem of multi-objective joint routing and scheduling. Specifically, we formulate the multi-action problem in joint routing and scheduling as a Multi-Markov Decision Process (MMDP) and design a new reward function to optimize multiple objectives. When optimizing the learning agent, we introduce an A3C-based multi-strategy optimization algorithm (A3C-MSO) to train two sub-policies, including the queue operation policy and the node operation policy for assigning queue operations to nodes. Furthermore, we integrate a graph convolutional network (GCN) into the learning framework to capture the spatial characteristics of irregular network topologies and enhance the algorithm’s generalization ability. Extensive experimental results in different scenarios indicate that compared to the existing state-of-the-art mechanisms, the proposed mechanism has shown a 13% improvement in schedulability and an 18% enhancement in resource utilization. Particularly in high-load scenarios, the time cost of the proposed mechanism can be reduced by up to 40.5%. Furthermore, results obtained on real industrial network topology instances indicate that the proposed learning strategies exhibit good generalization and effectiveness in large-scale scheduling instances.

Multi-agent Deep Reinforcement Learning for Cross-Layer Scheduling in Mobile Ad-Hoc Networks

ADSA: A Multi-path Transmission Scheduling Algorithm Based on Deep Reinforcement Learning in Vehicle Networks

Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks.

Multi-agent Reinforcement Learning-based User Scheduling and Resource Allocation in Cell-free Networks.

Lyapunov-guided Multi-Agent Reinforcement Learning for Delay-Sensitive Wireless Scheduling

Multi-Agent Reinforcement Learning for Multi-Cell Spectrum and Power Allocation

Packet Routing Against Network Congestion: A Deep Multi-agent Reinforcement Learning Approach

Traffic-driven Spectrum and Power Allocation Via Scalable Multi-Agent Reinforcement Learning

Energy-Aware MPTCP Scheduling in Heterogeneous Wireless Networks Using Multi-Agent Deep Reinforcement Learning Techniques

Learning to Schedule Communication in Multi-agent Reinforcement Learning

Load-Aware Distributed Resource Allocation for MF-TDMA Ad Hoc Networks: A Multi-Agent DRL Approach.

Decentralized Routing and Radio Resource Allocation in Wireless Ad Hoc Networks via Graph Reinforcement Learning

Cross-Layer Rate Control and Dynamic Scheduling in Mobile Ad Hoc Networks

A Multi-Policy Deep Reinforcement Learning Approach for Multi-Objective Joint Routing and Scheduling in Deterministic Networks

DeepCC: Multi-Agent Deep Reinforcement Learning Congestion Control for Multi-Path TCP Based on Self-Attention

Deep Reinforcement Learning for Distributed Dynamic Power Allocation in Wireless Networks.

Joint Optimization of Handover Control and Power Allocation Based on Multi-Agent Deep Reinforcement Learning

Intelligent Deep Reinforcement Learning-Based Scheduling in Relay-Based HetNets

MACC: Cross-Layer Multi-Agent Congestion Control with Deep Reinforcement Learning

MA-CC: Cross-Layer Congestion Control via Multi-agent Reinforcement Learning

Multi-Agent Deep Reinforcement Learning Based Adaptive User Association In Heterogeneous Networks