Abstract:The emerging technology of reconfigurable intelligent surfaces (RISs) is provisioned as an enabler of smart wireless environments, offering a highly scalable, low-cost, hardware-efficient, and almost energy-neutral solution for dynamic control of the propagation of electromagnetic signals over the wireless medium, ultimately providing increased environmental intelligence for diverse operation objectives. One of the major challenges with the envisioned dense deployment of RISs in such reconfigurable radio environments is the efficient configuration of multiple metasurfaces with limited, or even the absence of, computing hardware. In this article, we consider multiuser and multi-RIS-empowered wireless systems and present a thorough survey of the online machine learning approaches for the orchestration of their various tunable components. Focusing on the sum-rate maximization as a representative design objective, we present a comprehensive problem formulation based on deep reinforcement learning (DRL). We detail the correspondences among the parameters of the wireless system and the DRL terminology, and devise generic algorithmic steps for the artificial neural network training and deployment while discussing their implementation details. Further practical considerations for multi-RIS-empowered wireless communications in the sixth-generation (6G) era are presented along with some key open research challenges. Different from the DRL-based status quo, we leverage the independence between the configuration of the system design parameters and the future states of the wireless environment, and present efficient multiarmed bandits approaches, whose resulting sum-rate performances are numerically shown to outperform random configurations, while being sufficiently close to the conventional deep $Q$ network (DQN) algorithm, but with lower implementation complexity.

Scalable Multi-Agent Learning for Situationally-Aware Multiple-Access and Grant-Free Transmissions

SMART: Situationally-Aware Multi-Agent Reinforcement Learning-Based Transmissions

Compressive Massive Random Access for Massive Machine-Type Communications (mMTC).

Delay-Aware Massive Random Access for Machine-Type Communications Via Hierarchical Stochastic Learning

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Multi-Agent Reinforcement Learning-Based Decentralized Spectrum Access in Vehicular Networks with Emergent Communication

Grant-free Rateless Multiple Access: A Novel Massive Access Scheme for Internet of Things.

Massive Access For Machine-Type Communications In Backhaul-Constrained Heterogeneous Networks

Continual MARL-assisted Traffic-Adaptive GC-MAC Protocol for Task-Driven Directional MANETs

Multi-Agent Reinforcement Learning-Based Distributed Channel Access for Next Generation Wireless Networks

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

Mean-field MARL-based Priority-Aware CSMA/CA Strategy in Large-Scale MANETs

Multi-Carrier Rateless Multiple Access: A Novel Protocol for Dynamic Massive Access

Online Multi-Agent Reinforcement Learning for Multiple Access in Wireless Networks

Learning Random Access Schemes for Massive Machine-Type Communication with MARL

TinyQMIX: Distributed Access Control for mMTC via Multi-agent Reinforcement Learning

Resilient Massive Access for SAGIN: A Deep Reinforcement Learning Approach

Multi-Agent Reinforcement Learning for Multi-Cell Spectrum and Power Allocation

Scalable Joint Learning of Wireless Multiple-Access Policies and their Signaling

AI-Assisted Low Information Latency Wireless Networking

Load-Aware Distributed Resource Allocation for MF-TDMA Ad Hoc Networks: A Multi-Agent DRL Approach.