Abstract:With the rapid advancement of the Internet of Things (IoT), there is a global surge in network traffic. Software-Defined Networks (SDNs) provide a holistic network perspective, facilitating software-based traffic analysis, and are more suitable to handle dynamic loads than a traditional network. The standard SDN architecture control plane has been designed for a single controller or multiple distributed controllers; however, a logically centralized single controller faces severe bottleneck issues. Most proposed solutions in the literature are based on the static deployment of multiple controllers without the consideration of flow fluctuations and traffic bursts, which ultimately leads to a lack of load balancing among controllers in real time, resulting in increased network latency. Moreover, some methods addressing dynamic controller mapping in multi-controller SDNs consider load fluctuation and latency but face controller placement problems. Earlier, we proposed priority scheduling and congestion control algorithm (eSDN) and dynamic mapping of controllers for dynamic SDN (dSDN) to address this issue. However, the future growth of IoT is unpredictable and potentially exponential; to accommodate this futuristic trend, we need an intelligent solution to handle the complexity of growing heterogeneous devices and minimize network latency. Therefore, this paper continues our previous research and proposes temporal deep Q learning in the dSDN controller. A Temporal Deep Q learning Network (tDQN) serves as a self-learning reinforcement-based model. The agent in the tDQN learns to improve decision-making for switch-controller mapping through a reward–punish scheme, maximizing the goal of reducing network latency during the iterative learning process. Our approach—tDQN—effectively addresses dynamic flow mapping and latency optimization without increasing the number of optimally placed controllers. A multi-objective optimization problem for flow fluctuation is formulated to divert the traffic to the best-suited controller dynamically. Extensive simulation results with varied network scenarios and traffic show that the tDQN outperforms traditional networks, eSDNs, and dSDNs in terms of throughput, delay, jitter, packet delivery ratio, and packet loss.

OptRISQL: Towards Performance Improvement of Time-Varying IoT Networks Using Q-Learning

A deep reinforcement learning-based multi-optimality routing scheme for dynamic IoT networks

A novel routing optimization strategy based on reinforcement learning in perception layer networks

Artificial Intelligence Inspired Transmission Scheduling in Cognitive Vehicular Communications and Networks.

Q-Learning-Based High Credibility and Stability Routing Algorithm for Internet of Medical Things

A Practical AoI Scheduler in IoT Networks with Relays

Reinforcement Learning-Enabled Cross-Layer Optimization for Low-Power and Lossy Networks under Heterogeneous Traffic Patterns

Resource Allocation in Uplink NOMA-IoT Networks: A Reinforcement-Learning Approach

Research on Optimal Intelligent Routing Algorithm for IoV with Machine Learning and Smart Contract

RL-IoT: Reinforcement Learning-Based Routing Approach for Cognitive Radio-Enabled IoT Communications

Distance Vector and Prominent Reliable Path Selection based Stochastic Routing in Distributed Internet of Things

A New Deep-Q-Learning-Based Transmission Scheduling Mechanism for the Cognitive Internet of Things

EOMR: An Energy-Efficient Optimal Multi-path Routing Protocol to Improve QoS in Wireless Sensor Network for IoT Applications

Energy-efficient deep Q-network: reinforcement learning for efficient routing protocol in wireless internet of things

Multi-objective intelligent clustering routing schema for internet of things enabled wireless sensor networks using deep reinforcement learning

A Temporal Deep Q Learning for Optimal Load Balancing in Software-Defined Networks

DQQS: Deep Reinforcement Learning-Based Technique for Enhancing Security and Performance in SDN-IoT Environments

Reliable paths prediction with intelligent data plane monitoring enabled reinforcement learning in SD-IoT

An IoT and Machine Learning‐based Routing Protocol for Reconfigurable Engineering Application

An Improved Congestion-Controlled Routing Protocol for IoT Applications in Extreme Environments

Qoe Driven Resource Allocation In Massive Iot: A Deep Reinforcement Learning Approach