Abstract:As the demand for mobile data traffic continues to grow, offloading data traffic to unlicensed spectrum is a promising approach that can relieve the pressure on cellular systems. Therefore, it is an urgent need to propose an unlicensed spectrum access method to guarantee the harmonious and efficient coexistence between cellular network technologies such as LTE and incumbent users such as WiFi in the unlicensed spectrum. However, existing coexistence schemes such as licensed assisted access (LAA) and LTE-unlicensed (LTE-U) still suffer from inefficient spectrum utilization and unsatisfactory fairness. In the paper, we formulate the optimization problem of the unlicensed spectrum access among multiple small bases (SBSs) as a game, and then solve the Nash Equilibrium (NE) with cooperative and distributed multi-agent deep reinforcement learning (MADRL). Specifically, a two level access framework for the coexistence scenario, which consists of feedback cycle and executive cycle, is first proposed, and then the key elements of MADRL including state, action, reward and Q-network are designed in detail based on the proposed access framework. To overcome the problems of learning divergence and prohibitive computation overhead in the coexistence scenario with multiple SBSs due to the non-stability phenomena, we adopt the mean field technology to solve the NE, which can simplify the process of solving NE by converting the interaction of an agent with the remaining multiple agents into an action with the average effect of them. Simulation results show that 1) the proposed algorithm can overcome the learning divergence problem and converge to the NE quickly, and 2) the proposed algorithm can achieve the bi-objective optimization of total throughput and fairness of the coexistence network, and can achieve better performance in terms of throughput and fairness compared with the baseline methods such as Cat-4 LBT, Cooperative LBT and Random schemes.

Interference Coordination for Autonomous Small Cell Networks Based on Distributed Learning.

Decentralized Learning Based Indoor Interference Mitigation for 5G-and-beyond Systems

Reinforcement Learning-Based Interference Control for Ultra-Dense Small Cells.

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

AI-Aided Downlink Interference Control in Dense Interference-Aware Drone Small Cells Networks

Multi-Agent Deep Reinforcement Learning for Computation Offloading and Interference Coordination in Small Cell Networks

An Effective Inter-Cell Interference Coordination Scheme for Heterogeneous Network

Distributed Unsupervised Learning for Interference Management in Integrated Sensing and Communication Systems

Interference Coordination in Full-Duplex HetNet with Large-Scale Antenna Arrays.

A Distributed Learning-based Interference Coordination and Resource Allocation Scheme.

Interference Coordination for Co-Channel Deployed Macrocell and Small Cell Cluster

Deep Reinforcement Learning for Distributed Dynamic MISO Downlink-Beamforming Coordination

Energy Efficient Beamforming for Small Cell Systems: A Distributed Learning and Multicell Coordination Approach

Radio Resource Management for Ultra-Dense Smallcell Networks: A Hybrid Spectrum Reuse Approach.

Reinforcement Learning Based Interference Coordination for Port Communications

Joint interference management in ultra-dense small cell networks: A multi-dimensional coordination

A distributed coalition formation scheme for interference management in dense small cell networks

Joint Cooperative Clustering and Power Control for Energy-Efficient Cell-Free XL-MIMO with Multi-Agent Reinforcement Learning

Joint Interference Alignment and Power Control for Dense Networks Via Deep Reinforcement Learning

Distributed Multi-Agent Interference Coordination in Native AI Enabled Multi-Cell Networks for 6G

Intelligent Access to Unlicensed Spectrum: A Mean Field Based Deep Reinforcement Learning Approach