Abstract:The past few years has witnessed a surge of interest towards deep reinforcement learning (Deep RL) in computer networks. With extraordinary ability of feature extraction, Deep RL has the potential to re-engineer the fundamental resource allocation problems in networking without relying on pre-programmed models or assumptions about dynamic environments. However, such black-box systems suffer from poor robustness, showing high performance variance and poor tail performance. In this work, we propose a unified Teacher-Student learning framework that harnesses rich domain knowledge to improve robustness. The domain-specific algorithms, less performant but more trustable than Deep RL, play the role of teachers providing advice at critical states; the student neural network is steered to maximize the expected reward as usual and mimic the teacher's advice meanwhile. The Teacher-Student method comprises of three modules where the confidence check module locates wrong decisions and risky decisions, the reward shaping module designs a new updating function to incentive the learning of student network, and the prioritized experience replay module to effectively utilize the advised actions. We further implement our Teacher-Student framework in existing video streaming (Pensieve), load balancing (DeepLB) and TCP congestion control (Aurora). Experimental results manifest that the proposed approach reduces the performance standard deviation of DeepLB by 37%; it improves the 90th, 95th and 99th tail performance of Pensieve by 7.6%, 8.8%, 10.7% respectively; and it accelerates the rate of growth of Aurora by 2x at the initial stage, and achieves a more stable performance in dynamic environments.

SOHO-FL: A Fast Reconvergent Intra-Domain Routing Scheme Using Federated Learning

Fast Convergence for Federated Learning in OFDMA Systems

Fast-Convergence Reinforcement Learning for Routing in LEO Satellite Networks

Deep-Reinforcement-Learning-Based Intelligent Routing Strategy for FANETs

Enabling High-Throughput Routing for LEO Satellite Broadband Networks: A Flow-Centric Deep Reinforcement Learning Approach

A Hybrid Deep Reinforcement Learning Routing Method under Dynamic and Complex Traffic with Software Defined Networking

Federated Learning with Unsourced Random Access.

A deep reinforcement learning-based multi-optimality routing scheme for dynamic IoT networks

Topology Learning for Heterogeneous Decentralized Federated Learning Over Unreliable D2D Networks

Reinforcement Learning based Energy-Efficient Fast Routing for FANETs

A novel federated learning approach for routing optimisation in opportunistic IoT networks

Enabling Robust DRL-Driven Networking Systems Via Teacher-Student Learning

Enabling Scalable Routing in Software-Defined Networks With Deep Reinforcement Learning on Critical Nodes

Improving the Scalability of Deep Reinforcement Learning-Based Routing with Control on Partial Nodes

Convergence Analysis and System Design for Federated Learning over Wireless Networks.

Leveraging Domain Knowledge for Robust Deep Reinforcement Learning in Networking

Hybrid Intelligent Routing with Optimized Learning (HIROL) for Adaptive Routing Topology management in FANETs

A Deep Reinforcement Learning-based Routing Scheme with Two Modes for Dynamic Networks

Hierarchical Federated Learning through LAN-WAN Orchestration

Reinforcement Learning based Interconnection Routing for Adaptive Traffic Optimization