Abstract:With the rapid advancement of the Internet of Things (IoT), there is a global surge in network traffic. Software-Defined Networks (SDNs) provide a holistic network perspective, facilitating software-based traffic analysis, and are more suitable to handle dynamic loads than a traditional network. The standard SDN architecture control plane has been designed for a single controller or multiple distributed controllers; however, a logically centralized single controller faces severe bottleneck issues. Most proposed solutions in the literature are based on the static deployment of multiple controllers without the consideration of flow fluctuations and traffic bursts, which ultimately leads to a lack of load balancing among controllers in real time, resulting in increased network latency. Moreover, some methods addressing dynamic controller mapping in multi-controller SDNs consider load fluctuation and latency but face controller placement problems. Earlier, we proposed priority scheduling and congestion control algorithm (eSDN) and dynamic mapping of controllers for dynamic SDN (dSDN) to address this issue. However, the future growth of IoT is unpredictable and potentially exponential; to accommodate this futuristic trend, we need an intelligent solution to handle the complexity of growing heterogeneous devices and minimize network latency. Therefore, this paper continues our previous research and proposes temporal deep Q learning in the dSDN controller. A Temporal Deep Q learning Network (tDQN) serves as a self-learning reinforcement-based model. The agent in the tDQN learns to improve decision-making for switch-controller mapping through a reward–punish scheme, maximizing the goal of reducing network latency during the iterative learning process. Our approach—tDQN—effectively addresses dynamic flow mapping and latency optimization without increasing the number of optimally placed controllers. A multi-objective optimization problem for flow fluctuation is formulated to divert the traffic to the best-suited controller dynamically. Extensive simulation results with varied network scenarios and traffic show that the tDQN outperforms traditional networks, eSDNs, and dSDNs in terms of throughput, delay, jitter, packet delivery ratio, and packet loss.

preDQN-Based TAS Traffic Scheduling in Intelligence Endogenous Networks

TATA: Throughput-Aware TAsk Placement in Heterogeneous Stream Processing with Deep Reinforcement Learning

Optimizing Traffic Scheduling in Autonomous Vehicle Networks Using Machine Learning Techniques and Time-Sensitive Networking

DRLS: A Deep Reinforcement Learning Based Scheduler for Time-Triggered Ethernet

Deep Reinforcement Learning-Based Adaptive Scheduling for Wireless Time-Sensitive Networking

AI-based Dynamic Schedule Calculation in Time Sensitive Networks using GCN-TD3

DeepScheduler: Enabling Flow-Aware Scheduling in Time-Sensitive Networking.

DR-IS: Dynamic Response Incremental Scheduling in Time-Sensitive Network

TTDeep - Time-Triggered Scheduling for Real-Time Ethernet Via Deep Reinforcement Learning.

DNLight: Learning Efficient Evaluation for Traffic Signal Control

Tensor-Based Reinforcement Learning for Network Routing

Traffic Priority-Aware Multi-User Distributed Dynamic Spectrum Access: A Multi-Agent Deep RL Approach

Time-Triggered Scheduling for Time-Sensitive Networking with Preemption

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

Joint Routing and Scheduling With Cyclic Queuing and Forwarding for Time-Sensitive Networks

Start Time Planning for Cyclic Queuing and Forwarding in Time-Sensitive Networks

Distributed and Adaptive Traffic Engineering with Deep Reinforcement Learning

A Temporal Deep Q Learning for Optimal Load Balancing in Software-Defined Networks

Transit Signal Priority under Connected Vehicle Environment: Deep Reinforcement Learning Approach

Deep Reinforcement Learning for Router Selection in Network with Heavy Traffic

Enabling efficient routing for traffic engineering in SDN with Deep Reinforcement Learning