Abstract:Deterministic Networking (DetNet) is a highly predictable and controllable network technology. It provides low packet loss rate and bounded latency data transmission for applications through resource reservation and scheduling mechanisms. However, DetNet is a hybrid traffic system, and the resource reservation mechanism cannot guarantee the deterministic requirements as the number of diverse deterministic applications increases. As a result, there is an urgent need for an efficient and fine-grained scheduling mechanism to meet the deterministic and bounded latency requirements. In this paper, we propose a novel end-to-end multi-policy deep reinforcement learning framework for automatically learning multiple policies and addressing the problem of multi-objective joint routing and scheduling. Specifically, we formulate the multi-action problem in joint routing and scheduling as a Multi-Markov Decision Process (MMDP) and design a new reward function to optimize multiple objectives. When optimizing the learning agent, we introduce an A3C-based multi-strategy optimization algorithm (A3C-MSO) to train two sub-policies, including the queue operation policy and the node operation policy for assigning queue operations to nodes. Furthermore, we integrate a graph convolutional network (GCN) into the learning framework to capture the spatial characteristics of irregular network topologies and enhance the algorithm’s generalization ability. Extensive experimental results in different scenarios indicate that compared to the existing state-of-the-art mechanisms, the proposed mechanism has shown a 13% improvement in schedulability and an 18% enhancement in resource utilization. Particularly in high-load scenarios, the time cost of the proposed mechanism can be reduced by up to 40.5%. Furthermore, results obtained on real industrial network topology instances indicate that the proposed learning strategies exhibit good generalization and effectiveness in large-scale scheduling instances.

A Reinforcement Learning Approach for Multipath TCP Data Scheduling

Multipath TCP Meets Reinforcement Learning: A Novel Energy-Efficient Scheduling Approach in Heterogeneous Wireless Networks

A Dynamic Packet Scheduling Method for Multipath TCP in Heterogeneous Wireless Networks

ReLeS: A Neural Adaptive Multipath Scheduler Based on Deep Reinforcement Learning

Multi-attribute Aware Data Scheduling for Multipath TCP

URLM: Utilising Reinforcement Learning to Schedule Subflows in MPTCP

DeepCC: Multi-Agent Deep Reinforcement Learning Congestion Control for Multi-Path TCP Based on Self-Attention

SmartCC: A Reinforcement Learning Approach for Multipath TCP Congestion Control in Heterogeneous Networks

Deep Reinforcement Learning for Router Selection in Network with Heavy Traffic

Breaking the Inertial Thinking: Non-Blocking Multipath Congestion Control Based on the Single-Subflow Reinforcement Learning Model

RAN Information-Assisted TCP Congestion Control Using Deep Reinforcement Learning with Reward Redistribution

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Energy-Aware MPTCP Scheduling in Heterogeneous Wireless Networks Using Multi-Agent Deep Reinforcement Learning Techniques

A Survey on Congestion Control and Scheduling for Multipath TCP: Machine Learning vs Classical Approaches

EdAR: an Experience-Driven Multipath Scheduler for Seamless Handoff in Mobile Networks

Joint QoS-Aware Scheduling and Precoding for Massive MIMO Systems via Deep Reinforcement Learning

A Multi-Policy Deep Reinforcement Learning Approach for Multi-Objective Joint Routing and Scheduling in Deterministic Networks

Reinforcement Learning Based Scheduling for Heterogeneous UAV Networking.

An Online Learning Assisted Packet Scheduler for MPTCP in Mobile Networks

Multi-Agent DRL-based Multipath Scheduling for Video Streaming with QUIC

Deep reinforcement learning-based fountain coding for concurrent multipath transfer in high-speed railway networks