Abstract:The emerging technology of reconfigurable intelligent surfaces (RISs) is provisioned as an enabler of smart wireless environments, offering a highly scalable, low-cost, hardware-efficient, and almost energy-neutral solution for dynamic control of the propagation of electromagnetic signals over the wireless medium, ultimately providing increased environmental intelligence for diverse operation objectives. One of the major challenges with the envisioned dense deployment of RISs in such reconfigurable radio environments is the efficient configuration of multiple metasurfaces with limited, or even the absence of, computing hardware. In this article, we consider multiuser and multi-RIS-empowered wireless systems and present a thorough survey of the online machine learning approaches for the orchestration of their various tunable components. Focusing on the sum-rate maximization as a representative design objective, we present a comprehensive problem formulation based on deep reinforcement learning (DRL). We detail the correspondences among the parameters of the wireless system and the DRL terminology, and devise generic algorithmic steps for the artificial neural network training and deployment while discussing their implementation details. Further practical considerations for multi-RIS-empowered wireless communications in the sixth-generation (6G) era are presented along with some key open research challenges. Different from the DRL-based status quo, we leverage the independence between the configuration of the system design parameters and the future states of the wireless environment, and present efficient multiarmed bandits approaches, whose resulting sum-rate performances are numerically shown to outperform random configurations, while being sufficiently close to the conventional deep $Q$ network (DQN) algorithm, but with lower implementation complexity.

Cooperative Hierarchical Deep Reinforcement Learning based Joint Sleep, Power, and RIS Control for Energy-Efficient HetNet

Cooperative Hierarchical Deep Reinforcement Learning based Joint Sleep and Power Control in RIS-aided Energy-Efficient RAN

Hierarchical Reinforcement Learning for RIS-Assisted Energy-Efficient RAN

Deep Reinforcement Learning for Energy-Efficient Power Control in Heterogeneous Networks

Hierarchical Radio Resource Optimization for Heterogeneous Networks with Enhanced Inter-Cell Interference Coordination (Eicic)

Intelligent Cloud-Edge Collaborations Assisted Energy-Efficient Power Control in Heterogeneous Networks

Robust Max-Min Energy Efficiency for RIS-Aided HetNets with Distortion Noises.

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Lightweight Cloud-Edge Collaborations for Intelligent Power Control in Energy-Efficient Heterogeneous Networks

Secure Energy Efficiency in Communication Systems Based on Deep Learning

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

Deep Reinforcement Learning for Multi-Agent Power Control in Heterogeneous Networks

Energy Cooperation with Sleep Mechanism in Renewable Energy Assisted Cellular HetNets

Joint Interference Alignment and Power Control for Dense Networks Via Deep Reinforcement Learning

Robust Security Energy Efficiency Optimization for RIS-Aided Cell-Free Networks with Multiple Eavesdroppers.

A Hierarchical DRL Approach for Resource Optimization in Multi-RIS Multi-Operator Networks

Deep Reinforcement Learning based Modulation and Coding Scheme Selection in Cognitive Heterogeneous Networks

Collaborative Multi-BS Power Management for Dense Radio Access Network using Deep Reinforcement Learning

Intelligent Decision-Based Edge Server Sleep for Green Computing in MEC-Enabled IoV Networks

Deep Reinforcement Learning for Modulation and Coding Scheme Selection in Cognitive HetNets

Deep-Reinforcement-Learning-Based IRS for Cooperative Jamming Networks under Edge Computing