Abstract:This paper investigates the coexistence of the New Radio-based access to Unlicensed spectrum (NR-U) network and the Wireless Gigabit (WiGig) network in unlicensed millimeter-wave (mmWave) bands. To enable the NR-U network to achieve equitable and harmonious spectrum sharing with WiGig systems, we develop two new classes of user equipment (UE) scheduling schemes by exploiting the deep reinforcement learning (DRL) technique. Specifically, we first propose the distributed deep reinforcement learning scheduling (DeepDS) scheme, wherein multiple deep neural networks (DNNs) are used to make decisions for different panels in an independent fashion. Thereafter, to reduce the computational cost of adopting multiple DNNs, we design the centralized deep reinforcement learning scheduling (DeepCS) scheme that introduces the shared DNN framework to perform decisions for all panels in parallel at one time. The objective of both DeepDS and DeepCS is to maximize the total data rate of the NR-U network with as little interference to WiGig systems as possible, while satisfying the quality of service (QoS) requirement for each UE. We first formulate this problem into the constrained Markov decision process framework. To address the multi-constraint issue, we put forth a new DRL algorithm that incorporates the Lagrangian primal-dual optimization into the deep Q-network framework, referred to as adaptive multi-constraint deep Q-network (AMC-DQN). With AMC-DQN, both DeepDS and DeepCS can achieve their goals even without acquiring prior operations about the WiGig network. Simulation results show that compared with the state-of-the-art omniLBT and dirLBT, both DeepDS and DeepCS yield significant performance benefits in terms of the total network data rate. We also demonstrate the ability of DeepDS and DeepCS to satisfy the QoS requirements of different UEs and their robustness against various simulation setups. Furthermore, compared with DeepDS, DeepCS can save a large amount of computational cost although at the expense of a slightly lower data rate.

Reliable Cybertwin-Driven Concurrent Multipath Transfer with Deep Reinforcement Learning.

A Dynamic Packet Scheduling Method for Multipath TCP in Heterogeneous Wireless Networks

Deep reinforcement learning-based fountain coding for concurrent multipath transfer in high-speed railway networks

Multipath TCP Meets Reinforcement Learning: A Novel Energy-Efficient Scheduling Approach in Heterogeneous Wireless Networks

A Reinforcement Learning Approach for Multipath TCP Data Scheduling

SmartCC: A Reinforcement Learning Approach for Multipath TCP Congestion Control in Heterogeneous Networks

Learning-Based Real-Time Transmission Control for Multi-Path TCP Networks.

DeepPath: Multi-path Traffic Scheduling Algorithm for 5G High Dynamic Network

Cybertwin-Driven DRL-Based Adaptive Transmission Scheduling for Software Defined Vehicular Networks

Breaking the Inertial Thinking: Non-Blocking Multipath Congestion Control Based on the Single-Subflow Reinforcement Learning Model

ReLeS: A Neural Adaptive Multipath Scheduler Based on Deep Reinforcement Learning

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

DeepCC: Multi-Agent Deep Reinforcement Learning Congestion Control for Multi-Path TCP Based on Self-Attention

Multi-Band and Multi-Network Cooperative Transmission Algorithm for Wireless Resource Optimization

Bandwidth Scheduling For Multipath Tcp Based Concurrent Multipath Transfer

Reliable PPO-Based Concurrent Multipath Transfer for Time-Sensitive Applications

ADPSM: An Adaptive Delay-based Packet Splitting Mechanism for Concurrent Multipath Transfer over heterogeneous paths

A New Deep-Q-Learning-Based Transmission Scheduling Mechanism for the Cognitive Internet of Things

Multi-Objective Network Congestion Control Via Constrained Reinforcement Learning.

Deep Reinforcement Learning-Based Scheduling for NR-U/WiGig Coexistence in Unlicensed Mmwave Bands

A Survey on Congestion Control and Scheduling for Multipath TCP: Machine Learning vs Classical Approaches