Abstract:In this paper, we address the problem of distributed interference management of cognitive femtocells that share the same frequency range with macrocells (primary user) using distributed multi-agent Q-learning. We formulate and solve three problems representing three different Q-learning algorithms: namely, centralized, distributed and partially distributed power control using Q-learning (CPC-Q, DPC-Q and PDPC-Q). CPCQ, although not of practical interest, characterizes the global optimum. Each of DPC-Q and PDPC-Q works in two different learning paradigms: Independent (IL) and Cooperative (CL). The former is considered the simplest form for applying Qlearning in multi-agent scenarios, where all the femtocells learn independently. The latter is the proposed scheme in which femtocells share partial information during the learning process in order to strike a balance between practical relevance and performance. In terms of performance, the simulation results showed that the CL paradigm outperforms the IL paradigm and achieves an aggregate femtocells capacity that is very close to the optimal one. For the practical relevance issue, we evaluate the robustness and scalability of DPC-Q, in real time, by deploying new femtocells in the system during the learning process, where we showed that DPC-Q in the CL paradigm is scalable to large number of femtocells and more robust to the network dynamics compared to the IL paradigm

Cell Selection in Two-Tier Femtocell Networks Using Q-learning Algorithm

Cell Selection in Two-Tier Femtocell Networks with Open/closed Access Using Evolutionary Game

Cell selection in Femtocell network

QoS-aware energy-efficient power control in two-tier femtocell networks based on Q-learning

On Improving Energy Efficiency within Green Femtocell Networks: A Hierarchical Reinforcement Learning Approach

Improving energy efficiency in Green femtocell networks: A hierarchical reinforcement learning framework

Subchannel Allocation Scheme for Two-tire Femtocell Networks Based on Q-learning

A Cooperative Q-learning Approach for Real-time Power Allocation in Femtocell Networks

A Distributed User Association Method for LTE-U by Enabling Q-Learning in Minority Game.

Dynamic uplink/downlink configuration using Q-learning in femtocell networks

Q-Learning Based Network Selection for WCDMA/WLAN Heterogeneous Wireless Networks

Heterogeneous Network Selection Algorithm for Novel 5G Services Based on Evolutionary Game

Cell Selection in Heterogeneous Wireless Networks

Distributed Cell Selection in Heterogeneous Wireless Networks.

Multi-Agent Reinforcement Learning Based Unlicensed Resource Sharing for LTE-U Networks.

Q-Learning-Based Network Selection Strategy for Access Control in WLAN/WIMAX

A Distributed Network Selection Method Based on Minority Game for LTE in Unlicensed Bands.

Distributed interference management using Q-Learning in Cognitive Femtocell networks: New USRP-based Implementation

Approach for Power Allocation in Two-Tier Femtocell Networks Based on Robust Non-Cooperative Game.

BiLSTM Based Reinforcement Learning for Resource Allocation and User Association in LTE-U Networks

Q-Learning Based Selection Strategies for Load Balance and Energy Balance in Heterogeneous Networks