Abstract:To accommodate the explosive wireless traffics, massive multiple-input multiple-output (MIMO) is regarded as one of the key enabling technologies for next-generation communication systems. In massive MIMO cellular networks, coordinated beamforming (CBF), which jointly designs the beamformers of multiple base stations (BSs), is an efficient method to enhance the network performance. In this paper, we investigate the sum rate maximization problem in a massive MIMO mobile cellular network, where in each cell a multi-antenna BS serves multiple mobile users simultaneously via downlink beamforming. Although existing optimization-based CBF algorithms can provide near-optimal solutions, they require realtime and global channel state information (CSI), in addition to their high computation complexity. It is almost impossible to apply them in practical wireless networks, especially highly dynamic mobile cellular networks. Motivated by this, we propose a deep reinforcement learning based distributed dynamic coordinated beamforming (DDCBF) framework, which enables each BS to determine the beamformers with only local CSI and some historical information from other BSs.Besides, the beamformers can be calculated with a considerably lower computational complexity by exploiting neural networks and expert knowledge, i.e., a solution structure observed from the iterative procedure of the weighted minimum mean square error (WMMSE) algorithm. Moreover, we provide extensive numerical simulations to validate the effectiveness of the proposed DRL-based approach. With lower computational complexity and less required information, the results show that the proposed approach can achieve comparable performance to the centralized iterative optimization algorithms.

Reinforcement Learning for Scheduling and Mimo beam Selection using Caviar Simulations

Deep Reinforcement Learning Based on Location-Aware Imitation Environment for RIS-Aided Mmwave MIMO Systems

Learning-assisted User Scheduling and Beamforming for mmWave Vehicular Networks

Optimal User Scheduling in Multi Antenna System Using Multi Agent Reinforcement Learning

Deep Reinforcement Learning for Multi-user Massive MIMO with Channel Aging

A Deep Reinforcement Learning-Based Resource Scheduler for Massive MIMO Networks

Efficient Initial Access with Deep Reinforcement Learning Based Beam Sweeping in Wireless Cellular Communication Systems

Joint Deep Reinforcement Learning and Unfolding: Beam Selection and Precoding for Mmwave Multiuser MIMO with Lens Arrays

DRL-Based Sequential Scheduling for IRS-Assisted MIMO Communications

An MRL-Based Design Solution for RIS-Assisted MU-MIMO Wireless System under Time-Varying Channels

Deep Reinforcement Learning for Distributed Dynamic Coordinated Beamforming in Massive MIMO Cellular Networks

Deep Reinforcement Learning Based Dynamic Beam Selection in Dual-Band Communication Systems

Refined-Deep Reinforcement Learning for MIMO Bistatic Backscatter Resource Allocation

Actor-Critic Scheduling for Path-Aware Air-to-Ground Multipath Multimedia Delivery

Joint QoS-Aware Scheduling and Precoding for Massive MIMO Systems via Deep Reinforcement Learning

Joint Band Assignment and Beam Management using Hierarchical Reinforcement Learning for Multi-Band Communication

Delay Optimal Scheduling for Cognitive Radios with Cooperative Beamforming: A Structured Matrix-Geometric Method

Deep Reinforcement Learning Based End-to-End Multiuser Channel Prediction and Beamforming

Deep Reinforcement Learning based Joint Active and Passive Beamforming Design for RIS-Assisted MISO Systems

Multi-Agent Deep Reinforcement Learning Joint Beamforming for Slicing Resource Allocation

Joint Sub-Band and Transmission Rate Selection for Anti-Jamming Non-Contiguous Orthogonal Frequency Division Multiplexing System: An Upper Confidence Bound Based Reinforcement Learning Approach