Abstract:Cooperative spectrum sensing (CSS) technology has been widely studied to enhance the spectrum sharing efficiency spatially and temporally in cognitive radio networks (CRNs), where the secondary users (SUs) can opportunistically reuse the channels already licensed to the primary users (PUs) for transmission by sensing spectrum holes. SUs are endowed with the global awareness of channels state by cooperating with each other without sweeping across the whole frequency bands. Since the channels occupation of PUs changes dynamically, the accurate sensing and swift information sharing are crucial for CRNs. The paper proposes a multi-agent deep reinforcement learning (DRL) based CSS method to help SUs efficiently finding a vacant channel by the cooperation with their partners. 1) Two partner selection algorithms are proposed named as Reliable Partner CSS and Adaptive Partner CSS, respectively. For the former, the partner selection is facilitated based on the historical sensing accuracy of SUs. While the latter takes the comprehensive consideration of both the reliability and geographical distribution of SUs to further improve the sensing accuracy. 2) Multi-agent deep deterministic policy gradient (MADDPG) is adopted to resist the dynamically varying channels condition as well as the high-dimension solution space. With the feature of 'centralized training and decentralized execution', each SU learns to interact with the environment and select a vacant channel for transmission by its partial observation, which greatly reduces the communication overhead caused by the cooperative spectrum sensing. 3) Numerical simulation demonstrates the convergence and availability of the proposed algorithms. No matter Reliable Partner CSS or Adaptive Partner CSS, the sensing accuracy can be greatly enhanced comparing with other non-cooperative or centralized learning approaches. Besides, the attention mechanism is introduced to MADDPG for Adaptive Partner CSS to reveal the behavior of SUs by the visualization of attention weight, which helps to partially interpret the 'black box' issue of DRL.

Multi-Agent Cooperation-Based Deep Reinforcement Learning for Multisensor Perception Communication System in HSR Tunnel Scenario

Multi-agent deep reinforcement learning based multiple access for underwater cognitive acoustic sensor networks

Intelligent Resource Allocation for Train-to-Train Communication: A Multi-Agent Deep Reinforcement Learning Approach.

Distributed DRL for Device Access in Symbiotic Radio-Aided High-Speed Railway Networks.

Attention Enhanced Multi-Agent Reinforcement Learning for Cooperative Spectrum Sensing in Cognitive Radio Networks

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

When mmWave High-Speed Railway Networks Meet Reconfigurable Intelligent Surface: A Deep Reinforcement Learning Method

Deep Reinforcement Learning for Computation and Communication Resource Allocation in Multiaccess MEC Assisted Railway IoT Networks

Efficient Communications for Multi-Agent Reinforcement Learning in Wireless Networks

Cooperative multi-agent target searching: a deep reinforcement learning approach based on parallel hindsight experience replay

Deep Reinforcement Learning Coordinated Receiver Beamforming for Millimeter-Wave Train-ground Communications

Adaptive Traffic Signal Control for Large-Scale Scenario with Cooperative Group-based Multi-agent Reinforcement Learning

Cooperative Optimization of Traffic Signals and Vehicle Speed Using a Novel Multi-agent Deep Reinforcement Learning

Traffic Priority-Aware Multi-User Distributed Dynamic Spectrum Access: A Multi-Agent Deep RL Approach

Minimizing AoI in High-Speed Railway Mobile Networks: DQN-Based Methods

Dynamic Multichannel Sensing in Cognitive Radio: Hierarchical Reinforcement Learning

Deep reinforcement learning-based fountain coding for concurrent multipath transfer in high-speed railway networks

A multi‐agent deep reinforcement learning approach for traffic signal coordination

V2X User Scheduling for Collaborative Perception Based on Reinforcement Learning

Multi-agent deep reinforcement learning with centralized training and decentralized execution for transportation infrastructure management

A Cognitive Multi-Carrier Radar for Communication Interference Avoidance Via Deep Reinforcement Learning