Abstract:The most recent development of the Internet of Things brings massive timely sensitive and bursty data flows. Also, joint optimization on storage, computation, and communication is in need for multiaccess edge computing frameworks. The adaptive network control has been explored using deep reinforcement learning (RL), but it is not sufficient for bursty network traffic flows, especially when the network traffic pattern may change over time. We formulate the routing control in an environment with time-variant link delays as a Lyapunov optimization problem. We identify that there is a tradeoff between optimization performance and modeling accuracy when the propagation delays are included. We propose a novel deep RL (DRL)-based adaptive network routing method to tackle the issues mentioned above. A Lyapunov optimization technique is used to reduce the upper bound of the Lyapunov drift, improving queuing stability in networked systems. By modeling the network traffic pattern using the Markovian arrival process, we show that network routing problems can be modeled as Markov decision processes and value-iteration-based RL methods can be used to solve them. We design a blockchain-based protocol using proof of elapsed time consensus mechanism to ensure a trustworthy network statistics information exchange for the routing framework. Experiment results show that the proposed method can learn a routing policy and adapt to the changing environment. The proposed method outperforms the baseline backpressure method in multiple settings and converges faster than existing methods. Moreover, the DRL module can effectively learn a better estimation of the long-term Lyapunov drift and penalty functions, providing superior results in terms of the backlog size, end-to-end latency, age of information, and throughput. Furthermore, the blockchain-based network statistics exchange can provide the routing framework against malicious nodes. In addition, the proposed model performs well under various topologies, and thus can be used in general cases.

CMDRL: A Markovian Distributed Rate Limiting Algorithm in Cloud Networks

Finite-Time Convergence Rates of Decentralized Local Markovian Stochastic Approximation

TRCC: Transferable Congestion Control with Reinforcement Learning

"DRL + FL": an Intelligent Resource Allocation Model Based on Deep Reinforcement Learning for Mobile Edge Computing

Two-tiered Online Optimization of Region-wide Datacenter Resource Allocation via Deep Reinforcement Learning

ACC-RL: Adaptive Congestion Control Based on Reinforcement Learning in Power Distribution Networks with Data Centers

Consistent and Fine-Grained Rule Update with In-Network Control for Distributed Rate Limiting

A DRL-Driven Intelligent Optimization Strategy for Resource Allocation in Cloud-Edge-End Cooperation Environments

Traffic Management for Distributed Machine Learning in RDMA-enabled Data Center Networks.

Reliable and Efficient RAR-based Distributed Model Training in Computing Power Network

DMADRL: A Distributed Multi-agent Deep Reinforcement Learning Algorithm for Cognitive Offloading in Dynamic MEC Networks

Adaptive DRL-Based Task Scheduling for Energy-Efficient Cloud Computing

Transient Analysis of Clustered Multitask Diffusion RLS Algorithm

RAN Information-assisted TCP Congestion Control via DRL with Reward Redistribution

Adaptive and Robust Routing with Lyapunov-Based Deep RL in MEC Networks Enabled by Blockchains.

RAN Information-Assisted TCP Congestion Control Using Deep Reinforcement Learning with Reward Redistribution

ZiXia: A Reinforcement Learning Approach via Adjusted Ranking Reward for Internet Congestion Control

Deep-RL: Deep Reinforcement Learning for Marking-Aware Via Per-Port in Data Centers

RLCC: Practical Learning-based Congestion Control for the Internet

DRL-TAL: Deep Reinforcement Learning-Based Traffic-Aware Load Balancing in Data Center Networks

Multi-Objective Network Congestion Control Via Constrained Reinforcement Learning.