Abstract:The emerging technology of reconfigurable intelligent surfaces (RISs) is provisioned as an enabler of smart wireless environments, offering a highly scalable, low-cost, hardware-efficient, and almost energy-neutral solution for dynamic control of the propagation of electromagnetic signals over the wireless medium, ultimately providing increased environmental intelligence for diverse operation objectives. One of the major challenges with the envisioned dense deployment of RISs in such reconfigurable radio environments is the efficient configuration of multiple metasurfaces with limited, or even the absence of, computing hardware. In this article, we consider multiuser and multi-RIS-empowered wireless systems and present a thorough survey of the online machine learning approaches for the orchestration of their various tunable components. Focusing on the sum-rate maximization as a representative design objective, we present a comprehensive problem formulation based on deep reinforcement learning (DRL). We detail the correspondences among the parameters of the wireless system and the DRL terminology, and devise generic algorithmic steps for the artificial neural network training and deployment while discussing their implementation details. Further practical considerations for multi-RIS-empowered wireless communications in the sixth-generation (6G) era are presented along with some key open research challenges. Different from the DRL-based status quo, we leverage the independence between the configuration of the system design parameters and the future states of the wireless environment, and present efficient multiarmed bandits approaches, whose resulting sum-rate performances are numerically shown to outperform random configurations, while being sufficiently close to the conventional deep $Q$ network (DQN) algorithm, but with lower implementation complexity.

6G Fresnel Spot Beamfocusing using Large-Scale Metasurfaces: A Distributed DRL-Based Approach

Deep Learning-driven Adaptive Metasurface Transmission Focusing

Towards Near-Field 3D Spot Beamfocusing: Possibilities, Challenges, and Use-cases

Near-Field Spot Beamfocusing: A Correlation-Aware Transfer Learning Approach

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

Beam Focusing for Near-Field Multi-User Localization

Hybrid Beamforming for RIS-Empowered Multi-hop Terahertz Communications: A DRL-based Method

Self-adaptive deep reinforcement learning for THz beamforming with silicon metasurfaces in 6G communications

Symmetry Enhanced Network Architecture Search for Complex Metasurface Design

Achieving Beamfocusing via Two Separated Uniform Linear Arrays

Energy-efficient Beamforming for RISs-aided Communications: Gradient Based Meta Learning

DRL-Based Sidelobe Suppression for Multi-focus Reconfigurable Intelligent Surface

Fast MIMO Beamforming via Deep Reinforcement Learning for High Mobility mmWave Connectivity

Beam Focusing for Near-Field Multi-User MIMO Communications

Site-Specific Beam Alignment in 6G via Deep Learning

Physics-driven unsupervised deep learning network for programmable metasurface-based beamforming

6G Wireless Communications: From Far-Field Beam Steering to Near-Field Beam Focusing

Design of stacked intelligent metasurfaces with reconfigurable amplitude and phase for multiuser downlink beamforming

Near-Field Beam Tracking with Extremely Large Dynamic Metasurface Antennas

Advancing Ultra-Reliable 6G: Transformer and Semantic Localization Empowered Robust Beamforming in Millimeter-Wave Communications