Abstract:Device-to-device (D2D) communication has been regarded as a promising solution to alleviate the mobile traffic explosion problem for its capabilities of improving system data rate and resource utilization. A reconfigurable intelligent surface (RIS) aided mobile D2D communications framework is investigated, where the RIS is deployed to improve communication quality. As the transmission distance of D2D pairs changes, the mode selection for D2D pairs and the phase shift design for RIS is essential for mobile scenarios. Therefore, we formulate a joint optimization problem of mode selection, channel assignment, power allocation, and discrete phase shift selection to maximize the average sum data rate of D2D pairs. This problem is also constrained by the maximum transmit power and the minimum data rate requirements of users, where the latter is to guarantee the fairness of D2D pairs. We first reformulate the original sequential decision-making problem into a Markov game (MG) problem to solve the challenging optimization. Furthermore, a multi-agent deep reinforcement learning (MADRL) framework is proposed, in which multiple agents cooperatively determine the joint mode selection and resource allocation strategy. The proposed MADRL-based framework combines both the multi-pass deep Q-networks (MP-DQN) algorithm and the decaying DQN algorithm to solve the optimization problem. Specifically, we adopt the MP-DQN algorithm for D2D pairs to handle the hybrid discrete-continuous action space. Moreover, the decaying DQN algorithm is invoked by the RIS agent to select discrete phase shifts. Simulation results demonstrate that the proposed algorithm can converge under different cases. The proposed MADRL-based algorithm outperforms the combination algorithm of DQN and the deep deterministic policy gradient (DDPG) in terms of system performance. Moreover, it is also shown that the average sum data rate of D2D pairs can be significantly improved by deploying the RIS and further enhanced by increasing the number of reflecting elements (REs).

Multi-Agent DRL Approach to Two-Timescale Transmission for RIS-Aided MU-MISO Systems

A Deep Reinforcement Learning Approach to Two-Timescale Transmission for RIS-Aided Multiuser MISO Systems

Multi-Agent DRL Based Transmission Optimization for Distributed RIS-Aided Communication Systems

Joint User Scheduling and Precoding for RIS-Aided MU-MISO Systems: A MADRL Approach

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

Reconfigurable Intelligent Surface-Assisted Aerial-Terrestrial Communications Via Multi-Task Learning

An MRL-Based Design Solution for RIS-Assisted MU-MIMO Wireless System under Time-Varying Channels

Reconfigurable Intelligent Surface Assisted Multiuser MISO Systems Exploiting Deep Reinforcement Learning

Deep Reinforcement Learning Empowered Joint Mode Selection and Resource Allocation for RIS-aided D2D Communications

Multi-agent deep reinforcement learning (MADRL) meets multi-user MIMO systems

Reconfigurable Intelligent Surface-Enhanced Broadband OFDM Communication Based on Deep Reinforcement Learning

A Robust Deep Learning-Based Beamforming Design for RIS-Assisted Multiuser MISO Communications with Practical Constraints

Joint Beamforming in RIS-Assisted Multi-User Transmission Design: A Model-Driven Deep Reinforcement Learning Framework

Joint AP Scheduling and Precoding in RIS-Aided Distributed MIMO Networks: A Hierarchical DRL Framework

Deep Reinforcement Learning Based Power Minimization for RIS-Assisted MISO-OFDM Systems

Deep Reinforcement Learning based Downlink Beamforming and Phase Optimization for RIS-Aided Communication System

STAR-RIS-aided secure communications for MEC with delay/energy-constrained QoS using multi-agent deep reinforcement learning

Multi-agent Reinforcement Learning-based Joint Precoding and Phase Shift Optimization for RIS-aided Cell-Free Massive MIMO Systems

Piecewise-DRL: Joint Beamforming Optimization for RIS-Assisted MU-MISO Communication System

A DRL Approach for RIS-Assisted Full-Duplex UL and DL Transmission: Beamforming, Phase Shift and Power Optimization

Multi-User MISO with Stacked Intelligent Metasurfaces: A DRL-Based Sum-Rate Optimization Approach