Abstract:The emerging technology of reconfigurable intelligent surfaces (RISs) is provisioned as an enabler of smart wireless environments, offering a highly scalable, low-cost, hardware-efficient, and almost energy-neutral solution for dynamic control of the propagation of electromagnetic signals over the wireless medium, ultimately providing increased environmental intelligence for diverse operation objectives. One of the major challenges with the envisioned dense deployment of RISs in such reconfigurable radio environments is the efficient configuration of multiple metasurfaces with limited, or even the absence of, computing hardware. In this article, we consider multiuser and multi-RIS-empowered wireless systems and present a thorough survey of the online machine learning approaches for the orchestration of their various tunable components. Focusing on the sum-rate maximization as a representative design objective, we present a comprehensive problem formulation based on deep reinforcement learning (DRL). We detail the correspondences among the parameters of the wireless system and the DRL terminology, and devise generic algorithmic steps for the artificial neural network training and deployment while discussing their implementation details. Further practical considerations for multi-RIS-empowered wireless communications in the sixth-generation (6G) era are presented along with some key open research challenges. Different from the DRL-based status quo, we leverage the independence between the configuration of the system design parameters and the future states of the wireless environment, and present efficient multiarmed bandits approaches, whose resulting sum-rate performances are numerically shown to outperform random configurations, while being sufficiently close to the conventional deep $Q$ network (DQN) algorithm, but with lower implementation complexity.

Meta-Critic Reinforcement Learning for Intelligent Omnidirectional Surface Assisted Multi-User Communications

Meta-Critic Reinforcement Learning for IOS-Assisted Multi-User Communications in Dynamic Environments.

Wireless Channel Prediction for Multi-user Physical Layer with Deep Reinforcement Learning

Meta Learning for Meta-Surface: A Fast Beamforming Method for RIS-Assisted Communications Adapting to Dynamic Environments.

A Robust Deep Learning-Based Beamforming Design for RIS-Assisted Multiuser MISO Communications with Practical Constraints

Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning

Reconfigurable Intelligent Surface-Assisted Aerial-Terrestrial Communications Via Multi-Task Learning

A Dynamic Power Allocation Scheme in Power-Domain NOMA Using Actor-Critic Reinforcement Learning.

Deep Reinforcement Learning for Multi-user Massive MIMO with Channel Aging

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

Active RIS-aided EH-NOMA Networks: A Deep Reinforcement Learning Approach

Soft Actor-Critic-Based Multi-User Multi-TTI MIMO Precoding in Multi-Modal Real-Time Broadband Communications

Learning-Based Intelligent Reflecting Surface-Aided Cell-Free Massive MIMO Systems

Meta-Wall: Intelligent Omni-Surfaces Aided Multi-Cell MIMO Communications

Energy-efficient Beamforming for RISs-aided Communications: Gradient Based Meta Learning

Deep Reinforcement Learning based Joint Active and Passive Beamforming Design for RIS-Assisted MISO Systems

Multi-User MISO with Stacked Intelligent Metasurfaces: A DRL-Based Sum-Rate Optimization Approach

Reconfigurable Intelligent Surface Assisted Mobile Edge Computing With Heterogeneous Learning Tasks

Multi-User Adaptive Video Delivery over Wireless Networks: A Physical Layer Resource-Aware Deep Reinforcement Learning Approach

Robust Beamforming Design for IOS-Assisted Multi-User MISO Systems with Imperfect CSI

Collaborative Intelligent Reflecting Surface Networks with Multi-Agent Reinforcement Learning