Abstract:The emerging technology of reconfigurable intelligent surfaces (RISs) is provisioned as an enabler of smart wireless environments, offering a highly scalable, low-cost, hardware-efficient, and almost energy-neutral solution for dynamic control of the propagation of electromagnetic signals over the wireless medium, ultimately providing increased environmental intelligence for diverse operation objectives. One of the major challenges with the envisioned dense deployment of RISs in such reconfigurable radio environments is the efficient configuration of multiple metasurfaces with limited, or even the absence of, computing hardware. In this article, we consider multiuser and multi-RIS-empowered wireless systems and present a thorough survey of the online machine learning approaches for the orchestration of their various tunable components. Focusing on the sum-rate maximization as a representative design objective, we present a comprehensive problem formulation based on deep reinforcement learning (DRL). We detail the correspondences among the parameters of the wireless system and the DRL terminology, and devise generic algorithmic steps for the artificial neural network training and deployment while discussing their implementation details. Further practical considerations for multi-RIS-empowered wireless communications in the sixth-generation (6G) era are presented along with some key open research challenges. Different from the DRL-based status quo, we leverage the independence between the configuration of the system design parameters and the future states of the wireless environment, and present efficient multiarmed bandits approaches, whose resulting sum-rate performances are numerically shown to outperform random configurations, while being sufficiently close to the conventional deep $Q$ network (DQN) algorithm, but with lower implementation complexity.

Deep Reinforcement Learning for Secrecy Energy Efficiency Maximization in RIS-Assisted Networks

Optimizing secrecy energy efficiency in RIS-assisted MISO systems using deep reinforcement learning

DRL-based Secrecy Rate Optimization for RIS-Assisted Secure ISAC Systems

Secure Energy Efficiency in Communication Systems Based on Deep Learning

Deep Learning for Secure Transmission in Reconfigurable Intelligent Surface-Assisted Communications.

Robust Security Energy Efficiency Optimization for RIS-Aided Cell-Free Networks with Multiple Eavesdroppers.

Max-Min Security Energy Efficiency Optimization for RIS-Aided Cell-Free Networks

Deep Reinforcement Learning for Physical Layer Security Enhancement in Energy Harvesting Based Cognitive Radio Networks

Secrecy Energy Efficiency Maximization in RIS-Aided Wireless Networks

SEE Maximization in RIS-Aided Network with RSMA: A PPO-SCF Method

Active RIS-Assisted Secure Wireless Communication Based on DRL

A Deep Reinforcement Learning‐Based Physical Layer Security Enhancement Design for RIS‐Aided mmWave Communications With Practical Constraints

Deep Reinforcement Learning Based Intelligent Reflecting Surface for Secure Wireless Communications

Secure Beamforming Design for Reconfigurable Intelligent Surface Assisted Downlink Transmission

Secrecy Energy Efficiency Optimization for Reconfigurable Intelligent Surface-Aided Multiuser MISO Systems

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Secure Beamforming Design for RIS-Assisted Integrated Sensing and Communication Systems

Secure Transmission for Intelligent Reflecting Surface Assisted Communication with Deep Learning

Deep Reinforcement Learning based Downlink Beamforming and Phase Optimization for RIS-Aided Communication System

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

Sum-Rate Maximization in RIS-Aided Wireless-Powered D2D Communication Networks.