Abstract:As a promising technology in the 5G era, the artificial intelligence (AI) enabled Internet of controllable things (IoCT) is expected to be an integral part of heterogeneous networks (HetNets) in the future. However, the realization of ultra-reliable low-latency communications (URLLC) in IoCT communications underlaid HetNet has stringent quality of service (QoS) requirements, resulting in unprecedented challenges for existing wireless resource allocation methods. In this paper, we first describe a cellular HetNet model with uplink IoCT communications, then formulate a dynamic mixed-integer nonlinear programming (MINLP) resource allocation problem for maximizing the long-term average energy efficiency under URLLC requirements including reliability, latency, and transmission rate. To solve the problem, we propose a decentralized MADRL-based resource allocation algorithm with a decentralized partially observable Markov decision process (dec-POMDP) and a mixed-centralized-decentralized (MCD) framework to address the partial observability and the scalability issues, respectively. In addition, we design a reward function featuring the objective decomposition, baseline-guided scaling, and QoS violation penalty so that the agents are coordinated. Extensive experiments demonstrate the convergence, scalability, and robustness of the proposed algorithm. Besides, the proposed algorithm substantially outperforms conventional resource allocation methods and different agent communication mechanisms in terms of maximizing energy efficiency.

Deep Reinforcement Learning Based Resource Allocation for Heterogeneous Networks

Distributed Deep Reinforcement Learning-Based Spectrum and Power Allocation for Heterogeneous Networks

Deep Reinforcement Learning for User Association and Resource Allocation in Heterogeneous Cellular Networks

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Deep-Reinforcement-Learning-Based Resource Allocation in ultra-dense network.

Multi-Agent Reinforcement Learning Based Unlicensed Resource Sharing for LTE-U Networks.

Multi-agent Deep Reinforcement Learning Based Resource Management in Heterogeneous V2X Networks

Deep Reinforcement Learning for Multi-Agent Power Control in Heterogeneous Networks

Toward Scalable and Efficient Hierarchical Deep Reinforcement Learning for 5G RAN Slicing

Deep Reinforcement Learning for Distributed Uncoordinated Cognitive Radios Resource Allocation

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

Deep Reinforcement Learning for Distributed and Uncoordinated Cognitive Radios Resource Allocation

Deep Multi-Agent Reinforcement Learning for Resource Allocation in D2D Communication Underlaying Cellular Networks

Deep Multiagent Reinforcement-Learning-Based Resource Allocation for Internet of Controllable Things

Spectrum-Energy-Efficient Mode Selection and Resource Allocation for Heterogeneous V2X Networks: A Federated Multi-Agent Deep Reinforcement Learning Approach

A Self-Organizing Resource Allocation Strategy Based on Q-learning Approach in Ultra-Dense Networks

Joint Optimization of Handover Control and Power Allocation Based on Multi-Agent Deep Reinforcement Learning

A Power Allocation Scheme Based on Deep Reinforcement Learning in HetNets.

Distributed Two-tier DRL Framework for Cell-Free Network: Association, Beamforming and Power Allocation.

Multi-Agent Deep Reinforcement Learning Based Resource Allocation for Ultra-Reliable Low-Latency Internet of Controllable Things

Deep Q-Network for User Association in Heterogeneous Cellular Networks.