Abstract:Device-to-device (D2D) communication has been regarded as a promising solution to alleviate the mobile traffic explosion problem for its capabilities of improving system data rate and resource utilization. A reconfigurable intelligent surface (RIS) aided mobile D2D communications framework is investigated, where the RIS is deployed to improve communication quality. As the transmission distance of D2D pairs changes, the mode selection for D2D pairs and the phase shift design for RIS is essential for mobile scenarios. Therefore, we formulate a joint optimization problem of mode selection, channel assignment, power allocation, and discrete phase shift selection to maximize the average sum data rate of D2D pairs. This problem is also constrained by the maximum transmit power and the minimum data rate requirements of users, where the latter is to guarantee the fairness of D2D pairs. We first reformulate the original sequential decision-making problem into a Markov game (MG) problem to solve the challenging optimization. Furthermore, a multi-agent deep reinforcement learning (MADRL) framework is proposed, in which multiple agents cooperatively determine the joint mode selection and resource allocation strategy. The proposed MADRL-based framework combines both the multi-pass deep Q-networks (MP-DQN) algorithm and the decaying DQN algorithm to solve the optimization problem. Specifically, we adopt the MP-DQN algorithm for D2D pairs to handle the hybrid discrete-continuous action space. Moreover, the decaying DQN algorithm is invoked by the RIS agent to select discrete phase shifts. Simulation results demonstrate that the proposed algorithm can converge under different cases. The proposed MADRL-based algorithm outperforms the combination algorithm of DQN and the deep deterministic policy gradient (DDPG) in terms of system performance. Moreover, it is also shown that the average sum data rate of D2D pairs can be significantly improved by deploying the RIS and further enhanced by increasing the number of reflecting elements (REs).

Deep Reinforcement Learning Based Mode Selection and Power Allocation Scheme for Convergent Power-Efficient Network.

Collaborative Multi-BS Power Management for Dense Radio Access Network using Deep Reinforcement Learning

Deep Reinforcement Learning Based Multicast Mode Selection for SFN

A Dynamic Power Allocation Scheme in Power-Domain NOMA Using Actor-Critic Reinforcement Learning.

Energy-Efficient Mode Selection and Resource Allocation for D2D-Enabled Heterogeneous Networks: A Deep Reinforcement Learning Approach

Power Allocation in Multi-User Cellular Networks: Deep Reinforcement Learning Approaches

Energy Efficient Transmission in Underlay CR-NOMA Networks Enabled by Reinforcement Learning

Deep Reinforcement Learning for Distributed Dynamic Power Allocation in Wireless Networks.

Deep Reinforcement Learning for Energy-Efficient Power Control in Heterogeneous Networks

Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks.

Deep Reinforcement Learning Based Power Allocation for D2D Network

Deep Reinforcement Learning Method for Energy Efficient Resource Allocation in Next Generation Wireless Networks.

Power Allocation in Multi-Cell Networks Using Deep Reinforcement Learning

Deep Reinforcement Learning for Energy Saving in Radio Access Network

Joint Mode Selection and Power Adaptation for D2D Communication with Reinforcement Learning.

A General DRL-based Optimization Framework of User Association and Power Control for HetNet.

Deep Reinforcement Learning Empowered Joint Mode Selection and Resource Allocation for RIS-aided D2D Communications

Deep Reinforcement Learning for Joint Channel Selection and Power Control in D2D Networks

Power Allocation for Device-to-Multi-Device Enabled HetNets - A Deep Reinforcement Learning Approach.

Energy-Efficient Ultra-Dense Network with Deep Reinforcement Learning

Hybrid Centralized-Distributed Resource Allocation Based on Deep Reinforcement Learning for Cooperative D2D Communications