Abstract:As the demand for mobile data traffic continues to grow, offloading data traffic to unlicensed spectrum is a promising approach that can relieve the pressure on cellular systems. Therefore, it is an urgent need to propose an unlicensed spectrum access method to guarantee the harmonious and efficient coexistence between cellular network technologies such as LTE and incumbent users such as WiFi in the unlicensed spectrum. However, existing coexistence schemes such as licensed assisted access (LAA) and LTE-unlicensed (LTE-U) still suffer from inefficient spectrum utilization and unsatisfactory fairness. In the paper, we formulate the optimization problem of the unlicensed spectrum access among multiple small bases (SBSs) as a game, and then solve the Nash Equilibrium (NE) with cooperative and distributed multi-agent deep reinforcement learning (MADRL). Specifically, a two level access framework for the coexistence scenario, which consists of feedback cycle and executive cycle, is first proposed, and then the key elements of MADRL including state, action, reward and Q-network are designed in detail based on the proposed access framework. To overcome the problems of learning divergence and prohibitive computation overhead in the coexistence scenario with multiple SBSs due to the non-stability phenomena, we adopt the mean field technology to solve the NE, which can simplify the process of solving NE by converting the interaction of an agent with the remaining multiple agents into an action with the average effect of them. Simulation results show that 1) the proposed algorithm can overcome the learning divergence problem and converge to the NE quickly, and 2) the proposed algorithm can achieve the bi-objective optimization of total throughput and fairness of the coexistence network, and can achieve better performance in terms of throughput and fairness compared with the baseline methods such as Cat-4 LBT, Cooperative LBT and Random schemes.

Multi-Agent Deep Reinforcement Learning for Resource Allocation in the Multi-Objective HetNet

A Hybrid Optimization and Deep RL Approach for Resource Allocation in Semi-GF NOMA Networks

Multi-Agent Deep Reinforcement Learning-Based Partial Task Offloading and Resource Allocation in Edge Computing Environment

Multi-Agent Deep Reinforcement Learning Based Resource Allocation for Ultra-Reliable Low-Latency Internet of Controllable Things

DDPG with Transfer Learning and Meta Learning Framework for Resource Allocation in Underlay Cognitive Radio Network

Multi-Agent Reinforcement Learning Based Unlicensed Resource Sharing for LTE-U Networks.

Intelligent Access to Unlicensed Spectrum: A Mean Field Based Deep Reinforcement Learning Approach

A collaborative optimization strategy for computing offloading and resource allocation based on multi-agent deep reinforcement learning

Joint Optimization of Handover Control and Power Allocation Based on Multi-Agent Deep Reinforcement Learning

Multi-Agent Reinforcement Learning for Multi-Cell Spectrum and Power Allocation

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Multi-agent deep reinforcement learning (MADRL) meets multi-user MIMO systems

Multi-Objective Energy-Efficient Resource Allocation for Multi-RAT Heterogeneous Networks

Deep Reinforcement Learning for Multi-Agent Power Control in Heterogeneous Networks

5G Multi-Slices Bi-Level Resource Allocation by Reinforcement Learning

Mobility-Aware Centralized Reinforcement Learning for Dynamic Resource Allocation in HetNets

Multi-objective Resource Allocation for 5G Using Hierarchical Reinforcement Learning

Multi-Agent Reinforcement Learning for Network Selection and Resource Allocation in Heterogeneous multi-RAT Networks

Multi-agent deep reinforcement learning for end—edge orchestrated resource allocation in industrial wireless networks

Computation Offloading and Resource Allocation in NOMA-MEC: A Deep Reinforcement Learning Approach

Spectrum-Energy-Efficient Mode Selection and Resource Allocation for Heterogeneous V2X Networks: A Federated Multi-Agent Deep Reinforcement Learning Approach