Abstract:The emerging technology of reconfigurable intelligent surfaces (RISs) is provisioned as an enabler of smart wireless environments, offering a highly scalable, low-cost, hardware-efficient, and almost energy-neutral solution for dynamic control of the propagation of electromagnetic signals over the wireless medium, ultimately providing increased environmental intelligence for diverse operation objectives. One of the major challenges with the envisioned dense deployment of RISs in such reconfigurable radio environments is the efficient configuration of multiple metasurfaces with limited, or even the absence of, computing hardware. In this article, we consider multiuser and multi-RIS-empowered wireless systems and present a thorough survey of the online machine learning approaches for the orchestration of their various tunable components. Focusing on the sum-rate maximization as a representative design objective, we present a comprehensive problem formulation based on deep reinforcement learning (DRL). We detail the correspondences among the parameters of the wireless system and the DRL terminology, and devise generic algorithmic steps for the artificial neural network training and deployment while discussing their implementation details. Further practical considerations for multi-RIS-empowered wireless communications in the sixth-generation (6G) era are presented along with some key open research challenges. Different from the DRL-based status quo, we leverage the independence between the configuration of the system design parameters and the future states of the wireless environment, and present efficient multiarmed bandits approaches, whose resulting sum-rate performances are numerically shown to outperform random configurations, while being sufficiently close to the conventional deep $Q$ network (DQN) algorithm, but with lower implementation complexity.

Deep Reinforcement Learning-Driven Reconfigurable Intelligent Surface-Assisted Radio Surveillance with a Fixed-Wing UAV.

Deep reinforcement learning for IRS-assisted UAV covert communications

Energy Harvesting Reconfigurable Intelligent Surface for UAV Based on Robust Deep Reinforcement Learning

Learning-Based Robust and Secure Transmission for Reconfigurable Intelligent Surface Aided Millimeter Wave UAV Communications

RIS-Assisted Jamming Rejection and Path Planning for UAV-Borne IoT Platform: A New Deep Reinforcement Learning Framework

A deep reinforcement learning framework and its implementation for UAV-aided covert communication

Robust Secure UAV Communications with the Aid of Reconfigurable Intelligent Surfaces

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Covert UAV Data Transmission Via Semantic Communication: A DRL-Driven Joint Position and Power Optimization Method

Deep reinforcement learning aided secure UAV communications in the presence of moving eavesdroppers

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

3D-Trajectory and Phase-Shift Design for RIS-Assisted UAV Systems Using Deep Reinforcement Learning

Reconfigurable Intelligent Surface Aided Secure UAV Communications

Joint 3D trajectory and phase shift optimization via deep reinforcement learning for RIS-assisted UAV communication systems

UAV Detection Using Reinforcement Learning

Intercepting Unauthorized Aerial Robots in Controlled Airspace Using Reinforcement Learning

A Deep Reinforcement Learning Approach for Autonomous Reconfigurable Intelligent Surfaces

Optimal Design for Trajectory and Phase-Shift in UAV-Mounted-RIS Communications with Reinforcement Learning

Dynamic resource allocation in IRS-assisted UAV wideband cognitive radio networks: A DDQN-TD3 approach

UAV-Mounted Multi-Functional RIS for Combating Eavesdropping in Wireless Networks

Radio Resource Management for Cellular-Connected UAV: A Learning Approach