Abstract:With the rapid advancement of the Internet of Things (IoT), there is a global surge in network traffic. Software-Defined Networks (SDNs) provide a holistic network perspective, facilitating software-based traffic analysis, and are more suitable to handle dynamic loads than a traditional network. The standard SDN architecture control plane has been designed for a single controller or multiple distributed controllers; however, a logically centralized single controller faces severe bottleneck issues. Most proposed solutions in the literature are based on the static deployment of multiple controllers without the consideration of flow fluctuations and traffic bursts, which ultimately leads to a lack of load balancing among controllers in real time, resulting in increased network latency. Moreover, some methods addressing dynamic controller mapping in multi-controller SDNs consider load fluctuation and latency but face controller placement problems. Earlier, we proposed priority scheduling and congestion control algorithm (eSDN) and dynamic mapping of controllers for dynamic SDN (dSDN) to address this issue. However, the future growth of IoT is unpredictable and potentially exponential; to accommodate this futuristic trend, we need an intelligent solution to handle the complexity of growing heterogeneous devices and minimize network latency. Therefore, this paper continues our previous research and proposes temporal deep Q learning in the dSDN controller. A Temporal Deep Q learning Network (tDQN) serves as a self-learning reinforcement-based model. The agent in the tDQN learns to improve decision-making for switch-controller mapping through a reward–punish scheme, maximizing the goal of reducing network latency during the iterative learning process. Our approach—tDQN—effectively addresses dynamic flow mapping and latency optimization without increasing the number of optimally placed controllers. A multi-objective optimization problem for flow fluctuation is formulated to divert the traffic to the best-suited controller dynamically. Extensive simulation results with varied network scenarios and traffic show that the tDQN outperforms traditional networks, eSDNs, and dSDNs in terms of throughput, delay, jitter, packet delivery ratio, and packet loss.

Data-Driven Self-Learning Controller Design Approach for Power-Aware IoT Devices based on Double Q-Learning Strategy

Combining Data-Driven and Event-Driven for Online Learning Predictive Control in Power Converters

Heuristic action execution for energy efficient charge-sustaining control of connected hybrid vehicles with model-free double Q-learning

Supervisory Control of the Hybrid Off-Highway Vehicle for Fuel Economy Improvement Using Predictive Double Q-learning with Backup Models

A Research on Aero-engine Control Based on Deep Q Learning

Designing a Fuzzy Q-Learning Power Energy System Using Reinforcement Learning

A deep reinforcement learning approach to energy management control with connected information for hybrid electric vehicles

A Holistic Power Optimization Approach for Microgrid Control Based on Deep Reinforcement Learning

Optimization strategy based on deep reinforcement learning for home energy management

Model Free Deep Deterministic Policy Gradient Controller for Setpoint Tracking of Non-minimum Phase Systems

An Online Home Energy Management System Using Q-Learning and Deep Q-Learning

Data-Driven Control and Learning Systems

Federated Double Deep Q-learning for Joint Delay and Energy Minimization in IoT networks

Safety-Enhanced Self-Learning for Optimal Power Converter Control

An adaptive hierarchical energy management strategy for hybrid electric vehicles combining heuristic domain knowledge and data-driven deep reinforcement learning

Reinforcement Learning-Based Energy Management for Fuel Cell Electrical Vehicles Considering Fuel Cell Degradation

Position control of a digital electrohydraulic system with limited sensory data using double deep q-network controller

A Temporal Deep Q Learning for Optimal Load Balancing in Software-Defined Networks

Demand Charge Control for Energy-intensive Enterprises Based on Deep Reinforcement Learning

Towards self-learning control of HVAC systems with the consideration of dynamic occupancy patterns: Application of model-free deep reinforcement learning

CDDPG: A Deep-Reinforcement-Learning-Based Approach for Electric Vehicle Charging Control