Abstract:This paper investigates the network-assisted full-duplex (NAFD) cell-free millimeter-wave (mmWave) networks, where the distribution of the transmitting access points (T-APs) and receiving access points (R-APs) across distinct geographical locations mitigates cross-link interference, facilitating the attainment of a truly flexible duplex mode. To curtail deployment expenses and power consumption for mmWave band operations, each AP incorporates a hybrid digital-analog structure encompassing precoder/combiner functions. However, this incorporation introduces processing intricacies within channel estimation and precoding/combining design. In this paper, we first present a hybrid multiple-input multiple-output (MIMO) processing framework and derive explicit expressions for both uplink and downlink achievable rates. Then we formulate a power allocation problem to maximize the weighted bidirectional sum rates. To tackle this non-convex problem, we develop a collaborative multi-agent deep reinforcement learning (MADRL) algorithm called multi-agent twin delayed deep deterministic policy gradient (MATD3) for NAFD cell-free mmWave networks. Specifically, given the tightly coupled nature of both uplink and downlink power coefficients in NAFD cell-free mmWave networks, the MATD3 algorithm resolves such coupled conflicts through an interactive learning process between agents and the environment. Finally, the simulation results validate the effectiveness of the proposed channel estimation methods within our hybrid MIMO processing paradigm, and demonstrate that our MATD3 algorithm outperforms both multi-agent deep deterministic policy gradient (MADDPG) and conventional power allocation strategies.

Spectrum-efficient user grouping and resource allocation based on deep reinforcement learning for mmWave massive MIMO-NOMA systems

Joint Deep Reinforcement Learning and Unfolding: Beam Selection and Precoding for Mmwave Multiuser MIMO with Lens Arrays

Joint QoS-Aware Scheduling and Precoding for Massive MIMO Systems via Deep Reinforcement Learning

Deep Learning Based Radio Resource Management in NOMA Networks: User Association, Subchannel and Power Allocation

Downlink Non-Orthogonal Multiple Access Power Allocation Algorithm Based on Double Deep Q Network for Ensuring User's Quality of Service

A Dynamic Power Allocation Scheme in Power-Domain NOMA Using Actor-Critic Reinforcement Learning.

Deep reinforcement learning based joint cooperation clustering and downlink power control for cell-free massive MIMO

Network-Assisted Full-Duplex Cell-Free mmWave Networks: Hybrid MIMO Processing and Multi-Agent DRL-Based Power Allocation

Energy-efficient access point clustering and power allocation in cell-free massive MIMO networks: a hierarchical deep reinforcement learning approach

Deep Unfolding for Energy-Efficient Resource Allocation in Mmwave Networks with Multi-Connectivity

Self-Attention DDPG for Multi-Beam Combining in Mmwave MIMO Systems.

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Deep Learning Based Joint Beam Selection and Precoding Design for Mmwave Systems with Lens Arrays

DDPG with Transfer Learning and Meta Learning Framework for Resource Allocation in Underlay Cognitive Radio Network

Situation-Aware Resource Allocation for Multi-Dimensional Intelligent Multiple Access: A Proactive Deep Learning Framework

Deep Reinforcement Learning Based Multidimensional Resource Management for Energy Harvesting Cognitive NOMA Communications

Deep Reinforcement Learning for Multi-user Massive MIMO with Channel Aging

Fast MIMO Beamforming via Deep Reinforcement Learning for High Mobility mmWave Connectivity

Downlink Power Control for Cell-Free Massive MIMO with Deep Reinforcement Learning

DRL-Assisted Resource Allocation for NOMA-MEC Offloading with Hybrid SIC

Self-attention reinforcement learning for multi-beam combining in mmWave 3D-MIMO systems