Abstract:This paper investigates a new class of carrier-sense multiple access (CSMA) protocols that employ deep reinforcement learning (DRL) techniques, referred to as carrier-sense deep-reinforcement learning multiple access (CS-DLMA). The goal of CS-DLMA is to enable efficient and equitable spectrum sharing among a group of co-located heterogeneous wireless networks. Existing CSMA protocols, such as the medium access control (MAC) of WiFi, are designed for a homogeneous network in which all nodes adopt the same protocol. Such protocols suffer from severe performance degradation in a heterogeneous environment where there are nodes adopting other MAC protocols. CS-DLMA aims to circumvent this problem by making use of DRL. In particular, this paper adopts alpha-fairness as the general objective of CS-DLMA. With alpha-fairness, CS-DLMA can achieve a range of different objectives when coexisting with other MACs by changing the value of alpha. A salient feature of CS-DLMA is that it can achieve these objectives without knowing the coexisting MACs through a learning process based on DRL. The underpinning DRL technique in CS-DLMA is deep Q-network (DQN). However, the conventional DQN algorithms are not suitable for CS-DLMA due to their uniform time-step assumption. In CSMA protocols, time steps are non-uniform in that the time duration required for carrier sensing is smaller than the duration of data transmission. This paper introduces a non-uniform time-step formulation of DQN to address this issue. Our simulation results show that CS-DLMA can achieve the general alpha-fairness objective when coexisting with TDMA, ALOHA, and WiFi protocols by adjusting its own transmission strategy. Interestingly, we also find that CS-DLMA is more Pareto efficient than other CSMA protocols when coexisting with WiFi.

Q Learning-based Access-Control Algorithm for CDMA/WLAN Heterogeneous Networks

Q-Learning-Based Network Selection Strategy for Access Control in WLAN/WIMAX

Q-Learning Based Network Selection for WCDMA/WLAN Heterogeneous Wireless Networks

A User-Preference-Aware Terminal-Controlled Access Network Selection Scheme in Heterogeneous Wireless Networks.

Multi-agent Q-learning Based Admission Control Mechanism in Heterogeneous Wireless Networks for Multiple Services.

Air-Ground Cooperative Access Control Algorithm Based on Q-Learning

Q-learning based Cognitive Wireless Network access Mechanism

Multi-Agent Reinforcement Learning Based Unlicensed Resource Sharing for LTE-U Networks.

QoS-oriented media access control using reinforcement learning for next-generation WLANs

A Distributed Call Admission Control and Network Selection Scheme for Hybrid CDMA-OFDMA Networks

Non-Uniform Time-Step Deep Q-Network for Carrier-Sense Multiple Access in Heterogeneous Wireless Networks.

Q-learning based cognitive network access scheme

Q-Learning-Based Spectrum Access for Multimedia Transmission Over Cognitive Radio Networks

A Distributed User Association Method for LTE-U by Enabling Q-Learning in Minority Game.

Research on SNR-based Access Algorithm in CDMA/WLAN Heterogeneous Networks

LTE-U and Wi-Fi Coexistence Algorithm Based on Q-Learning in Multi-Channel.

A Centralized Multi-User Anti-Composite Intelligent Interference Algorithm Based on Improved Q-Learning

Intelligent Network Selection Algorithm for Multiservice Users in 5G Heterogeneous Network System: Nash Q-Learning Method.

A Self-Organizing Resource Allocation Strategy Based on Q-learning Approach in Ultra-Dense Networks

Joint Call Admission Control Algorithm for Multimedia Services in Heterogeneous Wireless Networks

Centralized Channel and Power Allocation for Cognitive Radio Networks: A Q-learning Solution