Abstract:The emerging technology of reconfigurable intelligent surfaces (RISs) is provisioned as an enabler of smart wireless environments, offering a highly scalable, low-cost, hardware-efficient, and almost energy-neutral solution for dynamic control of the propagation of electromagnetic signals over the wireless medium, ultimately providing increased environmental intelligence for diverse operation objectives. One of the major challenges with the envisioned dense deployment of RISs in such reconfigurable radio environments is the efficient configuration of multiple metasurfaces with limited, or even the absence of, computing hardware. In this article, we consider multiuser and multi-RIS-empowered wireless systems and present a thorough survey of the online machine learning approaches for the orchestration of their various tunable components. Focusing on the sum-rate maximization as a representative design objective, we present a comprehensive problem formulation based on deep reinforcement learning (DRL). We detail the correspondences among the parameters of the wireless system and the DRL terminology, and devise generic algorithmic steps for the artificial neural network training and deployment while discussing their implementation details. Further practical considerations for multi-RIS-empowered wireless communications in the sixth-generation (6G) era are presented along with some key open research challenges. Different from the DRL-based status quo, we leverage the independence between the configuration of the system design parameters and the future states of the wireless environment, and present efficient multiarmed bandits approaches, whose resulting sum-rate performances are numerically shown to outperform random configurations, while being sufficiently close to the conventional deep $Q$ network (DQN) algorithm, but with lower implementation complexity.

On Deep Learning for Radio Resource Management in A Non-stationary Radio Environment

DDPG with Transfer Learning and Meta Learning Framework for Resource Allocation in Underlay Cognitive Radio Network

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

Deep Learning based Wireless Resource Allocation with Application to Vehicular Networks

Deep Reinforcement Learning for Distributed Uncoordinated Cognitive Radios Resource Allocation

Deep Reinforcement Learning for Distributed and Uncoordinated Cognitive Radios Resource Allocation

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Learning to Branch: Accelerating Resource Allocation in Wireless Networks

Model-based Deep Learning for Wireless Resource Allocation in RSMA Communications Systems

Deep Learning for Radio-based Human Sensing: Recent Advances and Future Directions

Deep Learning for Radio Resource Allocation in Multi-Cell Networks

Deep Reinforcement Learning for Radio Resource Allocation in NOMA-based Remote State Estimation

Radio Resource Management for Cellular-Connected UAV: A Learning Approach

Graph Neural Networks for Scalable Radio Resource Management: Architecture Design and Theoretical Analysis

Learning to optimize: Training deep neural networks for wireless resource management

Meta-Learning Empowered Graph Neural Networks for Radio Resource Management

Application of deep neural network and deep reinforcement learning in wireless communication

Reinforcement-Learning-Based Robust Resource Management for Multi-Radio Systems

+2cmFlexible Resource Management in High-throughput Satellite Communication Systems: A Two-stage Machine Learning Framework

Graph Reinforcement Learning for Radio Resource Allocation