Abstract:The emerging technology of reconfigurable intelligent surfaces (RISs) is provisioned as an enabler of smart wireless environments, offering a highly scalable, low-cost, hardware-efficient, and almost energy-neutral solution for dynamic control of the propagation of electromagnetic signals over the wireless medium, ultimately providing increased environmental intelligence for diverse operation objectives. One of the major challenges with the envisioned dense deployment of RISs in such reconfigurable radio environments is the efficient configuration of multiple metasurfaces with limited, or even the absence of, computing hardware. In this article, we consider multiuser and multi-RIS-empowered wireless systems and present a thorough survey of the online machine learning approaches for the orchestration of their various tunable components. Focusing on the sum-rate maximization as a representative design objective, we present a comprehensive problem formulation based on deep reinforcement learning (DRL). We detail the correspondences among the parameters of the wireless system and the DRL terminology, and devise generic algorithmic steps for the artificial neural network training and deployment while discussing their implementation details. Further practical considerations for multi-RIS-empowered wireless communications in the sixth-generation (6G) era are presented along with some key open research challenges. Different from the DRL-based status quo, we leverage the independence between the configuration of the system design parameters and the future states of the wireless environment, and present efficient multiarmed bandits approaches, whose resulting sum-rate performances are numerically shown to outperform random configurations, while being sufficiently close to the conventional deep $Q$ network (DQN) algorithm, but with lower implementation complexity.

Multi-Agent Team Learning in Virtualized Open Radio Access Networks (O-RAN)

Multi Agent Team Learning in Disaggregated Virtualized Open Radio Access Networks (O-RAN)

Team Learning-Based Resource Allocation for Open Radio Access Network (O-RAN)

A Decentralized Pilot Assignment Algorithm for Scalable O-RAN Cell-Free Massive MIMO

A Multi-Agent Deep Reinforcement Learning Approach for RAN Resource Allocation in O-RAN

Multi-Task Learning as enabler for General-Purpose AI-native RAN

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Multi-Agent Reinforcement Learning for Multi-Cell Spectrum and Power Allocation

Multi-Agent Reinforcement Learning Based Unlicensed Resource Sharing for LTE-U Networks.

Distributed Learning Framework for eMBB-URLLC Multiplexing in Open Radio Access Networks

Multi-Agent Reinforcement Learning-Based Decentralized Spectrum Access in Vehicular Networks with Emergent Communication

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

Multi-Agent Reinforcement Learning for Network Selection and Resource Allocation in Heterogeneous multi-RAT Networks

Graph-Embedded Multi-Agent Learning for Smart Reconfigurable THz MIMO-NOMA Networks

User Access Control in Open Radio Access Networks: A Federated Deep Reinforcement Learning Approach

AoI-Oriented Resource Allocation for NOMA-Based Wireless Powered Cognitive Radio Networks Based on Multi-Agent Deep Reinforcement Learning

Mobile Cell-Free Massive MIMO with Multi-Agent Reinforcement Learning: A Scalable Framework

RLOps: Development Life-cycle of Reinforcement Learning Aided Open RAN

A Multi-Agent Reinforcement Learning Approach for Massive Access in NOMA-URLLC Networks

Deep reinforcement learning for RAN optimization and control

Adaptive Resource Allocation for Virtualized Base Stations in O-RAN with Online Learning