Abstract:The emerging technology of reconfigurable intelligent surfaces (RISs) is provisioned as an enabler of smart wireless environments, offering a highly scalable, low-cost, hardware-efficient, and almost energy-neutral solution for dynamic control of the propagation of electromagnetic signals over the wireless medium, ultimately providing increased environmental intelligence for diverse operation objectives. One of the major challenges with the envisioned dense deployment of RISs in such reconfigurable radio environments is the efficient configuration of multiple metasurfaces with limited, or even the absence of, computing hardware. In this article, we consider multiuser and multi-RIS-empowered wireless systems and present a thorough survey of the online machine learning approaches for the orchestration of their various tunable components. Focusing on the sum-rate maximization as a representative design objective, we present a comprehensive problem formulation based on deep reinforcement learning (DRL). We detail the correspondences among the parameters of the wireless system and the DRL terminology, and devise generic algorithmic steps for the artificial neural network training and deployment while discussing their implementation details. Further practical considerations for multi-RIS-empowered wireless communications in the sixth-generation (6G) era are presented along with some key open research challenges. Different from the DRL-based status quo, we leverage the independence between the configuration of the system design parameters and the future states of the wireless environment, and present efficient multiarmed bandits approaches, whose resulting sum-rate performances are numerically shown to outperform random configurations, while being sufficiently close to the conventional deep $Q$ network (DQN) algorithm, but with lower implementation complexity.

Automatic SPICE- Integrated Reinforcement Learning for Decap Optimization for EMI and Power Integrity

Physics-Assisted Deep Reinforcement Learning-Based TSV Array Design for Signal and Power Integrity Co-Optimization

Deep Reinforcement Learning-Based Pin Map Optimization of BGA Package for EMC

A Novel Deep Reinforcement Learning Methodology for EMI Mitigation in Multi-Scenario Pinmap Design

Pin Assignment Optimization of BGA Packages for EMI Mitigation Using Deep Reinforcement Learning

Deep Reinforcement Learning-Based Ground-Via Placement Optimization for EMI Mitigation

Decoupling Optimization for Complex PDN Structures Using Deep Reinforcement Learning

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

Transformer Network-based Reinforcement Learning Method for Power Distribution Network (PDN) Optimization of High Bandwidth Memory (HBM)

A Novel Deep Reinforcement Learning Methodology for EMI Mitigation in Multiscenario Pinmap Design

Reinforcement Learning for Antenna Selection and Optimization of Irregular Reconfigurable Intelligent Surfaces

IR-Aware ECO Timing Optimization Using Reinforcement Learning

Automated Design and Optimization of Distributed Filtering Circuits via Reinforcement Learning

Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications

Make Safe Decisions in Power System: Safe Reinforcement Learning Based Pre-decision Making for Voltage Stability Emergency Control

Techno-Economic Modeling and Safe Operational Optimization of Multi-Network Constrained Integrated Community Energy Systems

IRL-PM: An Inverse Reinforcement Learning-based Power Minimization in Multi-User MISO Networks.

Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications

Deep Reinforcement Learning for Optimizing Inverter Control: Fixed and Adaptive Gain Tuning Strategies for Power System Stability

"Deep Reinforcement Learning for Engineering Design Through Topology Optimization of Elementally Discretized Design Domains"

Learning Deterministic Policy with Target for Power Control in Wireless Networks