Abstract:The emerging technology of reconfigurable intelligent surfaces (RISs) is provisioned as an enabler of smart wireless environments, offering a highly scalable, low-cost, hardware-efficient, and almost energy-neutral solution for dynamic control of the propagation of electromagnetic signals over the wireless medium, ultimately providing increased environmental intelligence for diverse operation objectives. One of the major challenges with the envisioned dense deployment of RISs in such reconfigurable radio environments is the efficient configuration of multiple metasurfaces with limited, or even the absence of, computing hardware. In this article, we consider multiuser and multi-RIS-empowered wireless systems and present a thorough survey of the online machine learning approaches for the orchestration of their various tunable components. Focusing on the sum-rate maximization as a representative design objective, we present a comprehensive problem formulation based on deep reinforcement learning (DRL). We detail the correspondences among the parameters of the wireless system and the DRL terminology, and devise generic algorithmic steps for the artificial neural network training and deployment while discussing their implementation details. Further practical considerations for multi-RIS-empowered wireless communications in the sixth-generation (6G) era are presented along with some key open research challenges. Different from the DRL-based status quo, we leverage the independence between the configuration of the system design parameters and the future states of the wireless environment, and present efficient multiarmed bandits approaches, whose resulting sum-rate performances are numerically shown to outperform random configurations, while being sufficiently close to the conventional deep $Q$ network (DQN) algorithm, but with lower implementation complexity.

Deep reinforcement learning for automatic run-time adaptation of UWB PHY radio settings

Wireless Channel Prediction for Multi-user Physical Layer with Deep Reinforcement Learning

Removing the need for ground truth UWB data collection: self-supervised ranging error correction using deep reinforcement learning

DDPG with Transfer Learning and Meta Learning Framework for Resource Allocation in Underlay Cognitive Radio Network

Radio Resource Management for Cellular-Connected UAV: A Learning Approach

A Novel Sensor Scheduling Algorithm Based on Deep Reinforcement Learning for Bearing-Only Target Tracking in UWSNs

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

A Probabilistic Learning Approach to UWB Ranging Error Mitigation.

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

DQLEL: Deep Q-Learning for Energy-Optimized LoS/NLoS UWB Node Selection

Deep Learning for Ultra-Wideband Indoor Positioning

Multi-objective Deep Reinforcement Learning Based Joint Beamforming and Power Allocation in UAV Assisted Cellular Communication

Deep Learning Predictive Band Switching in Wireless Networks

Link characteristics study of ultra-wideband radios

Learning-Based Energy-Efficient Resource Management by Heterogeneous RF/VLC for Ultra-Reliable Low-Latency Industrial IoT Networks

Adaptive Contention Window Design using Deep Q-learning

Collaborative Multi-BS Power Management for Dense Radio Access Network using Deep Reinforcement Learning

Multi-Objective Deep Reinforcement Learning for 5G Base Station Placement to Support Localisation for Future Sustainable Traffic

Deep Reinforcement Learning for Beam Management in UAV Relay mmWave Networks

Multipath-Assisted Single-Anchor Localization Via Deep Variational Learning

Deep Reinforcement Learning for Power Control in Next-Generation WiFi Network Systems