Abstract:Time-triggered (TT) communication has long been studied in various industrial domains. The most challenging task of TT communication is to find a feasible schedule table. Network changes are inevitable due to the topology dynamics, varying data transmission requirements, etc. Once changes occur, the schedule table needs to be re-calculated in a timely manner. Solver-based methods and heuristic-based methods were proposed to solve this problem. However, solver-based methods employ integer linear programming (ILP) or satisfiability modulo theories (SMT) which have high computational complexity. On the other hand, heuristic-based methods are fast, but they need to be handcrafted based on the application characteristics. Thus, these methods are not general enough to work in complex scenarios especially in large networks.In this paper we propose DRLS – Deep Reinforcement Learning based TT Scheduling method. DRLS first trains an application or network specific scheduling agent offline. Then, the agent can be used for online scheduling of TT flows. However, off-the-shelf reinforcement learning techniques cannot handle the TT scheduling problem with typical complexity and scale. DRLS provides novel solutions to this challenge, including three key innovations: new representations for TT network adapted to various topologies, proper deep neural network (DNN) structures to capture network characteristics, and scalable reinforcement learning (RL) models to handle online TT scheduling. Comprehensive experiments have been conducted to compare the performance of DRLS and other methods (heuristics-based methods such as HLS, LS, HLD + LD, LS + LD, and ILP-based method). The results show that DRLS can not only adapt to specific network topologies, but also have better performance: runs much faster than ILP solver-based methods, and schedules about 23.9% more flows than traditional handcrafted heuristic-based methods.

Scheduling Real-Time Wireless Traffic: A Network-Aided Offline Reinforcement Learning Approach

DRLS: A Deep Reinforcement Learning Based Scheduler for Time-Triggered Ethernet

On the Power of Randomization for Scheduling Real-Time Traffic in Wireless Networks

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Deep Reinforcement Learning aided No-wait Flow Scheduling in Time-Sensitive Networks

Delay-Oriented Scheduling in 5G Downlink Wireless Networks Based on Reinforcement Learning With Partial Observations

Graph Attention Network–Based Deep Reinforcement Learning Scheduling Framework for In-Vehicle Time-Sensitive Networking

Load-Aware Distributed Resource Allocation for MF-TDMA Ad Hoc Networks: A Multi-Agent DRL Approach.

Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets

Adaptive Network Coding for Scheduling Real-time Traffic with Hard Deadlines

Deep Reinforcement Learning for Wireless Scheduling in Distributed Networked Control

Deep Reinforcement Learning-Based Adaptive Scheduling for Wireless Time-Sensitive Networking

Scalable Deep Reinforcement Learning-Based Online Routing for Multi-Type Service Requirements

Reinforcement Learning Based Congestion Control in a Real Environment.

Effective Scheduling Function Design in SDN through Deep Reinforcement Learning

A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning

Power-Aware Traffic Engineering Via Deep Reinforcement Learning

Multi-Objective Network Congestion Control Via Constrained Reinforcement Learning.

Buffer-Aware Wireless Scheduling Based On Deep Reinforcement Learning

Distributed Age-of-Information Scheduling with NOMA Via Deep Reinforcement Learning

AoI-Aware Resource Scheduling for Industrial IoT with Deep Reinforcement Learning