Abstract:Massive Ultra-Reliable and Low-Latency Communications (mURLLC), which integrates URLLC with massive access, is emerging as a new and important service class in the next generation (6G) for time-sensitive traffics and has recently received tremendous research attention. However, realizing efficient, delay-bounded, and reliable communications for a massive number of user equipments (UEs) in mURLLC, is extremely challenging as it needs to simultaneously take into account the latency, reliability, and massive access requirements. To support these requirements, the third generation partnership project (3GPP) has introduced enhanced grant-free (GF) transmission in the uplink (UL), with multiple active configured-grants (CGs) for URLLC UEs. With multiple CGs (MCG) for UL, UE can choose any of these grants as soon as the data arrives. In addition, non-orthogonal multiple access (NOMA) has been proposed to synergize with GF transmission to mitigate the serious transmission delay and network congestion problems. In this paper, we develop a novel learning framework for MCG-GF-NOMA systems with bursty traffic. We first design the MCG-GF-NOMA model by characterizing each CG using the parameters: the number of contention-transmission units (CTUs), the starting slot of each CG within a subframe, and the number of repetitions of each CG. Based on the model, the latency and reliability performances are characterized. We then formulate the MCG-GF-NOMA resources configuration problem taking into account three constraints. Finally, we propose a Cooperative Multi-Agent based Double Deep Q-Network (CMA-DDQN) algorithm to balance the allocations of the channel resources among MCGs so as to maximize the number of successful transmissions under the latency constraint. Our results show that the MCG-GF-NOMA framework can simultaneously improve the low latency and high reliability performances in massive URLLC.

A Multi-Agent Reinforcement Learning Approach for Massive Access in NOMA-URLLC Networks

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Multi-Agent Deep Reinforcement Learning for Massive Access in 5G and Beyond Ultra-Dense NOMA System

Deep Reinforcement Learning-Based Grant-Free NOMA Optimization for mURLLC

Massive Access in 5G and Beyond Ultra-Dense Networks: an MARL-Based NORA Scheme.

A General Deep Reinforcement Learning Framework for Grant-Free NOMA Optimization in mURLLC

Matching Combined Heterogeneous Multi-Agent Reinforcement Learning for Resource Allocation in NOMA-V2X Networks

Deep Reinforcement Learning for Uplink Scheduling in NOMA-URLLC Networks

Deep Multi-Agent Reinforcement Learning for Resource Allocation in NOMA-enabled MEC

Multi-Agent Reinforcement Learning Based Unlicensed Resource Sharing for LTE-U Networks.

Distributed Q-Learning Aided Uplink Grant-Free NOMA for Massive Machine-Type Communications.

Sparsity-Aware Intelligent Massive Random Access Control for Massive MIMO Networks: A Reinforcement Learning Based Approach

Secure Resource Allocation for NOMA PLC Networks with Multi-agent DRL

Resource Allocation for Multi-service NOMA System Based on Deep Reinforcement Learning

Deep-Reinforcement-Learning-Based Resource Allocation in ultra-dense network.

Optimization of Grant-Free NOMA With Multiple Configured-Grants for mURLLC

Spatial-Separable NOMA-Based Intelligent Hierarchical Fast Uplink Grant for Murllc over Cell-Free Networks

Multi-Agent Reinforcement Learning-Based Distributed Channel Access for Next Generation Wireless Networks

Joint Power Allocation and Channel Assignment for NOMA With Deep Reinforcement Learning

Dual Dynamic Scheduling for Hierarchical QoS in Uplink-NOMA: A Reinforcement Learning Approach

Resource Allocation in Uplink NOMA Systems: A Hybrid-Decision-Based Multi-Agent Deep Reinforcement Learning Approach