Abstract:Cooperative spectrum sensing (CSS) technology has been widely studied to enhance the spectrum sharing efficiency spatially and temporally in cognitive radio networks (CRNs), where the secondary users (SUs) can opportunistically reuse the channels already licensed to the primary users (PUs) for transmission by sensing spectrum holes. SUs are endowed with the global awareness of channels state by cooperating with each other without sweeping across the whole frequency bands. Since the channels occupation of PUs changes dynamically, the accurate sensing and swift information sharing are crucial for CRNs. The paper proposes a multi-agent deep reinforcement learning (DRL) based CSS method to help SUs efficiently finding a vacant channel by the cooperation with their partners. 1) Two partner selection algorithms are proposed named as Reliable Partner CSS and Adaptive Partner CSS, respectively. For the former, the partner selection is facilitated based on the historical sensing accuracy of SUs. While the latter takes the comprehensive consideration of both the reliability and geographical distribution of SUs to further improve the sensing accuracy. 2) Multi-agent deep deterministic policy gradient (MADDPG) is adopted to resist the dynamically varying channels condition as well as the high-dimension solution space. With the feature of 'centralized training and decentralized execution', each SU learns to interact with the environment and select a vacant channel for transmission by its partial observation, which greatly reduces the communication overhead caused by the cooperative spectrum sensing. 3) Numerical simulation demonstrates the convergence and availability of the proposed algorithms. No matter Reliable Partner CSS or Adaptive Partner CSS, the sensing accuracy can be greatly enhanced comparing with other non-cooperative or centralized learning approaches. Besides, the attention mechanism is introduced to MADDPG for Adaptive Partner CSS to reveal the behavior of SUs by the visualization of attention weight, which helps to partially interpret the 'black box' issue of DRL.

Deep Q-Network Based Dynamic Spectrum Access for Cognitive Networks with Limited Spectrum Sensing Capability SUs

Dynamic Spectrum Access Based on Deep Reinforcement Learning for Multiple Access in Cognitive Radio

Deep Reinforcement Learning for Simultaneous Sensing and Channel Access in Cognitive Networks

Dynamic Cooperative Spectrum Sensing Based on Deep Multi-User Reinforcement Learning

Listen-After-Collision Mechanism for Dynamic Spectrum Access Using Deep Q-Network with an Improved Thompson Sampling Algorithm

Dynamic Spectrum Access in Multi-Channel Cognitive Radio Networks.

DRL-based Dynamic Spectrum Access for Cognitive Radio Networks in Mobility Scenarios

An Asynchronous Interference-Aware Dynamic Spectrum Access Algorithm for Secondary Users.

Deep Reinforcement Learning-based Distributed Dynamic Spectrum Access in Multi-User Multi-channel Cognitive Radio Internet of Things Networks

Distributed Dynamic Spectrum Access for D2D Communications Underlying Cellular Networks Using Deep Reinforcement Learning

Deep Q-learning multiple networks based dynamic spectrum access with energy harvesting for green cognitive radio network

Resource Allocation and Access Strategy Selection for QoS Provisioning in Cognitive Networks

Distributive Dynamic Spectrum Access through Deep Reinforcement Learning: A Reservoir Computing Based Approach

Attention Enhanced Multi-Agent Reinforcement Learning for Cooperative Spectrum Sensing in Cognitive Radio Networks

DRL-based Underlay Dynamic Spectrum Access for Cognitive Satellite Networks under Spectrum Sensing Errors

Dynamic Multichannel Sensing in Cognitive Radio: Hierarchical Reinforcement Learning

A Novel Dynamic Spectrum Access Algorithm for Cognitive Radio Networks.

A New Distributed Dynamic Spectrum Access Model Based on DQN

Initial spectrum access control with QoS protection for active users in cognitive wireless networks

Multi-user Dynamic Spectrum Access Based on Reinforcement Learning

Deep Sensing for Next-Generation Dynamic Spectrum Sharing: More Than Detecting the Occupancy State of Primary Spectrum