Abstract:Machine learning (ML) has been empowering all aspects of the wireless communication system design, among which, the reinforcement learning (RL)-based approaches have attracted a lot of research attention since they can interact with the environment directly and learn from the collected experiences efficiently. In this paper, we propose a novel and efficient RL-based multi-beam combining scheme for future millimeter-wave (mmWave) three-dimensional (3D) multi-input multi-output (MIMO) communication systems. The proposed scheme does not require perfect channel state information (CSI) or precise user location information which both are generally difficult to obtain in practice, and well addresses the crucial challenge of computational complexity incurred by the extremely huge state and action spaces associated with multiple users, multiple paths, and multiple 3D beams. In particular, a self-attention deep deterministic policy gradient (DDPG)-based beam selection and combination framework is proposed to learn the 3D beamforming pattern without CSI adaptively. We aim to maximize the sum-rate of the mmWave 3D-MIMO system by optimizing the serving beam set and the corresponding combining weights for each user. To this end, the transformer is incorporated into the DDPG to obtain the global information of the input elements and capture the signal directions precisely, which leads to a near-optimal beamformer design. Simulation results verify the superiority of the proposed self-attention DDPG over conventional ML-based beamforming schemes in terms of sum-rate under various scenarios.

Reinforcement learning-based waveform optimization for MIMO multi-target detection

Weak Target Detection in Massive MIMO Radar via an Improved Reinforcement Learning Approach

Multi-Target Detection and Adaptive Waveform Design for Cognitive MIMO Radar

Joint Deep Reinforcement Learning and Unfolding: Beam Selection and Precoding for Mmwave Multiuser MIMO with Lens Arrays

Robust Waveform Design for Multi-Target Detection in Cognitive MIMO Radar

Self-Attention DDPG for Multi-Beam Combining in Mmwave MIMO Systems.

Deep Reinforcement Learning Based on Location-Aware Imitation Environment for RIS-Aided Mmwave MIMO Systems

MIMO Radar Waveform Design for Multipath Exploitation Using Deep Learning

Beam Selection for RIS-Enabled Terahertz Multi-User MIMO Systems via Multi-Task Learning

A Robust Deep Learning-Based Beamforming Design for RIS-Assisted Multiuser MISO Communications with Practical Constraints

Robust MIMO Beamforming for Cellular and Radar Coexistence

Deep Learning Based Joint Beam Selection and Precoding Design for Mmwave Systems with Lens Arrays

Colocated MIMO radar waveform design for angular SRL in the context of hypothesis testing

Self-attention reinforcement learning for multi-beam combining in mmWave 3D-MIMO systems

Cognitive Waveform Design for Dual-functional MIMO Radar-Communication Systems

Performance Analysis and Waveform Optimization of Integrated FD-MIMO Radar-Communication Systems

Reinforcement Learning for Beam Pattern Design in Millimeter Wave and Massive MIMO Systems

Deep Reinforcement Learning for Multi-user Massive MIMO with Channel Aging

Deep Reinforcement Learning based Joint Active and Passive Beamforming Design for RIS-Assisted MISO Systems

Learning to Select for MIMO Radar based on Hybrid Analog-Digital Beamforming

Waveform Design for MIMO DFRC Systems: Finer Sensing and Safer Communications