Abstract:Cellular-based networks are expected to offer connectivity for massive Internet of Things (mIoT) systems. However, their Random Access CHannel (RACH) procedure suffers from unreliability, due to the collision from the simultaneous massive access. Despite that this collision problem has been treated in existing RACH schemes, these schemes usually organize IoT devices' transmission and re-transmission along with fixed parameters, thus can hardly adapt to time-varying traffic patterns. Without adaptation, the RACH procedure easily suffers from high access delay, high energy consumption, or even access unavailability. With the goal of improving the RACH procedure, this paper targets to optimize the RACH procedure in real-time by maximizing a long-term hybrid multi-objective function, which consists of the number of access success devices, the average energy consumption, and the average access delay. To do so, we first optimize the long-term objective in the number of access success devices by using Deep Reinforcement Learning (DRL) algorithms for different RACH schemes, including Access Class Barring (ACB), Back-Off (BO), and Distributed Queuing (DQ). The converging capability and efficiency of different DRL algorithms including Policy Gradient (PG), Actor-Critic (AC), Deep Q-Network (DQN), and Deep Deterministic Policy Gradients (DDPG) are compared. Inspired by the results from this comparison, a decoupled learning strategy is developed to jointly and dynamically adapt the access control factors of those three access schemes. This decoupled strategy first leverage a Recurrent Neural Network (RNN) model to predict the real-time traffic values of the network environment, and then uses multiple DRL agents to cooperatively configure parameters of each RACH scheme.

Medium Access using Distributed Reinforcement Learning for IoTs with Low-Complexity Wireless Transceivers

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks.

Carrier-Sense Multiple Access for Heterogeneous Wireless Networks Using Deep Reinforcement Learning

DDPG with Transfer Learning and Meta Learning Framework for Resource Allocation in Underlay Cognitive Radio Network

Reinforcement Learning for Improved Random Access in Delay-Constrained Heterogeneous Wireless Networks

Wireless MAC Protocol Synthesis and Optimization with Multi-Agent Distributed Reinforcement Learning

Cross Layering using Reinforcement Learning in Cognitive Radio-based Industrial Internet of Ad-hoc Sensor Network

Distributed DRL for Device Access in Symbiotic Radio-Aided High-Speed Railway Networks.

Intelligent User Association for Symbiotic Radio Networks Using Deep Reinforcement Learning

Cooperative Multi-Agent Reinforcement Learning for Low-Level Wireless Communication

Deep Reinforcement Learning Optimal Transmission Algorithm for Cognitive Internet of Things with RF Energy Harvesting

Scalable Deep Reinforcement Learning for Routing and Spectrum Access in Physical Layer

Dynamic multiple access based on deep reinforcement learning for Internet of Things

Multi-agent Reinforcement Learning Based Distributed Dynamic Spectrum Access

Intelligent Resource Allocation in Dense LoRa Networks using Deep Reinforcement Learning

Online Multi-Agent Reinforcement Learning for Multiple Access in Wireless Networks

ML Framework for Wireless MAC Protocol Design

A Decoupled Learning Strategy for Massive Access Optimization in Cellular IoT Networks

Deep Reinforcement Learning for Distributed Dynamic Power Allocation in Wireless Networks.

Deep Reinforcement Learning MAC for Backscatter Communications Relying on Wi-Fi Architecture.