Abstract:In this paper, a novel framework is proposed to optimize the downlink multi-user communication of a millimeter wave base station, which is assisted by a reconfigurable intelligent reflector (IR). In particular, a channel estimation approach is developed to measure the channel state information (CSI) in real-time. First, for a perfect CSI scenario, the precoding transmission of the BS and the reflection coefficient of the IR are jointly optimized, via an iterative approach, so as to maximize the sum of downlink rates towards multiple users. Next, in the imperfect CSI scenario, a distributional reinforcement learning (DRL) approach is proposed to learn the optimal IR reflection and maximize the expectation of downlink capacity. In order to model the transmission rate's probability distribution, a learning algorithm, based on quantile regression (QR), is developed, and the proposed QR-DRL method is proved to converge to a stable distribution of downlink transmission rate. Simulation results show that, in the error-free CSI scenario, the proposed approach yields over 30% and 2-fold increase in the downlink sum-rate, compared with a fixed IR reflection scheme and direct transmission scheme, respectively. Simulation results also show that by deploying more IR elements, the downlink sum-rate can be significantly improved. However, as the number of IR components increases, more time is required for channel estimation, and the slope of increase in the IR-aided transmission rate will become smaller. Furthermore, under limited knowledge of CSI, simulation results show that the proposed QR-DRL method, which learns a full distribution of the downlink rate, yields a better prediction accuracy and improves the downlink rate by 10% for online deployments, compared with a Q-learning baseline.

Refined-Deep Reinforcement Learning for MIMO Bistatic Backscatter Resource Allocation

Deep Reinforcement Learning Based on Location-Aware Imitation Environment for RIS-Aided Mmwave MIMO Systems

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

DDPG with Transfer Learning and Meta Learning Framework for Resource Allocation in Underlay Cognitive Radio Network

Deep Reinforcement Learning based Joint Active and Passive Beamforming Design for RIS-Assisted MISO Systems

Deep Reinforcement Learning for Multi-user Massive MIMO with Channel Aging

Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning

An MRL-Based Design Solution for RIS-Assisted MU-MIMO Wireless System under Time-Varying Channels

Deep Contextual Bandit and Reinforcement Learning for IRS-Assisted MU-MIMO Systems

A Robust Deep Learning-Based Beamforming Design for RIS-Assisted Multiuser MISO Communications with Practical Constraints

Deep Reinforcement Learning for Distributed Dynamic MISO Downlink-Beamforming Coordination

Deep Reinforcement Learning for Backscatter Communications: Augmenting Intelligence in Future Internet of Things

Learning-Based Intelligent Reflecting Surface-Aided Cell-Free Massive MIMO Systems

Joint Task Offloading and Resource Allocation for Intelligent Reflecting Surface-Aided Integrated Sensing and Communication Systems Using Deep Reinforcement Learning Algorithm

Millimeter Wave Communications With an Intelligent Reflector: Performance Optimization and Distributional Reinforcement Learning

Deep Reinforcement Learning for Distributed Dynamic Coordinated Beamforming in Massive MIMO Cellular Networks

Deep Reinforcement Learning based Downlink Beamforming and Phase Optimization for RIS-Aided Communication System

A Practical Design Based on Deep Reinforcement Learning for RIS-Assisted Mmwave MIMO Systems

DRL-based Joint Beamforming and BS-RIS-UE Association Design for RIS-Assisted mmWave Networks

Deep Reinforcement Learning Based Intelligent Reflecting Surface Optimization for TDD MultiUser MIMO Systems

Deep Reinforcement Learning Based Intelligent Reflecting Surface Optimization for TDD Multi-User MIMO Systems