Abstract:Deterministic Networking (DetNet) is a highly predictable and controllable network technology. It provides low packet loss rate and bounded latency data transmission for applications through resource reservation and scheduling mechanisms. However, DetNet is a hybrid traffic system, and the resource reservation mechanism cannot guarantee the deterministic requirements as the number of diverse deterministic applications increases. As a result, there is an urgent need for an efficient and fine-grained scheduling mechanism to meet the deterministic and bounded latency requirements. In this paper, we propose a novel end-to-end multi-policy deep reinforcement learning framework for automatically learning multiple policies and addressing the problem of multi-objective joint routing and scheduling. Specifically, we formulate the multi-action problem in joint routing and scheduling as a Multi-Markov Decision Process (MMDP) and design a new reward function to optimize multiple objectives. When optimizing the learning agent, we introduce an A3C-based multi-strategy optimization algorithm (A3C-MSO) to train two sub-policies, including the queue operation policy and the node operation policy for assigning queue operations to nodes. Furthermore, we integrate a graph convolutional network (GCN) into the learning framework to capture the spatial characteristics of irregular network topologies and enhance the algorithm’s generalization ability. Extensive experimental results in different scenarios indicate that compared to the existing state-of-the-art mechanisms, the proposed mechanism has shown a 13% improvement in schedulability and an 18% enhancement in resource utilization. Particularly in high-load scenarios, the time cost of the proposed mechanism can be reduced by up to 40.5%. Furthermore, results obtained on real industrial network topology instances indicate that the proposed learning strategies exhibit good generalization and effectiveness in large-scale scheduling instances.

A Modified Long Short-Term Memory-Deep Deterministic Policy Gradient-Based Scheduling Method for Active Distribution Networks

MDP-Based Distribution Network Reconfiguration with Renewable Distributed Generation: Approximate Dynamic Programming Approach

Deep Reinforcement Learning Based Charging Scheduling for Household Electric Vehicles in Active Distribution Network

Deep Reinforcement Learning for Voltage Control and Renewable Accommodation Using Spatial-Temporal Graph Information

Deep Reinforcement Learning for Charging Scheduling of Electric Vehicles Considering Distribution Network Voltage Stability

Research on the multi-timescale optimal voltage control method for distribution network based on a DQN-DDPG algorithm

RL-ADN: A High-Performance Deep Reinforcement Learning Environment for Optimal Energy Storage Systems Dispatch in Active Distribution Networks

Data-Driven Online Energy Scheduling of a Microgrid Based on Deep Reinforcement Learning

A Multi-Policy Deep Reinforcement Learning Approach for Multi-Objective Joint Routing and Scheduling in Deterministic Networks

Two-timescale online coordinated schedule of active distribution network considering dynamic network reconfiguration via bi-level safe deep reinforcement learning

Deep Reinforcement Learning for Wireless Scheduling in Distributed Networked Control

Real-time autonomous dynamic reconfiguration based on deep learning algorithm for distribution network

A Foresight-Seeing and Transferable Optimization Method for Synergic Operation of Multiple Flexible Resources in Active Distribution Network

A Multi-Stage Optimization Approach for Active Distribution Network Scheduling Considering Coordinated Electrical Vehicle Charging Strategy

Optimizing Load Scheduling in Power Grids Using Reinforcement Learning and Markov Decision Processes

Application of a Deep Deterministic Policy Gradient Algorithm for Energy-Aimed Timetable Rescheduling Problem

Optimal Scheduling of the Active Distribution Network with Microgrids Considering Multi-Timescale Source-Load Forecasting

Optimal Bi-Level Bidding and Dispatching Strategy between Active Distribution Network and Virtual Alliances Using Distributed Robust Multi-Agent Deep Reinforcement Learning

Distributed flexible resource regulation strategy for residential communities based on deep reinforcement learning

Multi-timescale Active Distribution Network Scheduling Considering Demand Response and User Comprehensive Satisfaction