Abstract:The most recent development of the Internet of Things brings massive timely sensitive and bursty data flows. Also, joint optimization on storage, computation, and communication is in need for multiaccess edge computing frameworks. The adaptive network control has been explored using deep reinforcement learning (RL), but it is not sufficient for bursty network traffic flows, especially when the network traffic pattern may change over time. We formulate the routing control in an environment with time-variant link delays as a Lyapunov optimization problem. We identify that there is a tradeoff between optimization performance and modeling accuracy when the propagation delays are included. We propose a novel deep RL (DRL)-based adaptive network routing method to tackle the issues mentioned above. A Lyapunov optimization technique is used to reduce the upper bound of the Lyapunov drift, improving queuing stability in networked systems. By modeling the network traffic pattern using the Markovian arrival process, we show that network routing problems can be modeled as Markov decision processes and value-iteration-based RL methods can be used to solve them. We design a blockchain-based protocol using proof of elapsed time consensus mechanism to ensure a trustworthy network statistics information exchange for the routing framework. Experiment results show that the proposed method can learn a routing policy and adapt to the changing environment. The proposed method outperforms the baseline backpressure method in multiple settings and converges faster than existing methods. Moreover, the DRL module can effectively learn a better estimation of the long-term Lyapunov drift and penalty functions, providing superior results in terms of the backlog size, end-to-end latency, age of information, and throughput. Furthermore, the blockchain-based network statistics exchange can provide the routing framework against malicious nodes. In addition, the proposed model performs well under various topologies, and thus can be used in general cases.

Compact Learning Model for Dynamic Off-Chain Routing in Blockchain-Based IoT

A deep reinforcement learning-based multi-optimality routing scheme for dynamic IoT networks

Adaptive and Robust Routing with Lyapunov-Based Deep RL in MEC Networks Enabled by Blockchains.

RLChain: A DRL Approach for Blockchain Performance Optimization Towards IIoT

Joint Optimization and Scheduling of Computing and Caching Resource for Blockchain-Empowered IoT via CDRL Approach

Robust and energy-efficient RPL optimization algorithm with scalable deep reinforcement learning for IIoT

Secure Deep Reinforcement Learning for Dynamic Resource Allocation in Wireless MEC Networks

Reinforcement learning-based load balancing for heavy traffic Internet of Things

Deep Reinforcement Learning for Online Resource Allocation in IoT Networks: Technology, Development, and Future Challenges

Deep reinforcement learning approach for computation offloading in blockchain-enabled communications systems

GTD3-NET: A deep reinforcement learning-based routing optimization algorithm for wireless networks

NOMA-Enabled Cooperative Computation Offloading for Blockchain-Empowered Internet of Things: A Learning Approach

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

ERODRL: Efficient ISP Routing Optimization Via Deep Reinforcement Learning

DRLAR: A Deep Reinforcement Learning-Based Adaptive Routing Framework for Network-on-chips

A Distributed and Privacy-Aware High-Throughput Transaction Scheduling Approach for Scaling Blockchain.

Graph Neural Network-Enhanced Reinforcement Learning for Payment Channel Rebalancing

Proactive Look-Ahead Control of Transaction Flows for High-Throughput Payment Channel Network

A novel routing optimization strategy based on reinforcement learning in perception layer networks

Deep Reinforcement Learning Based Performance Optimization in Blockchain-Enabled Internet of Vehicle

Blockchain-Aided Network Resource Orchestration in Intelligent Internet of Things