Abstract:Antenna tuning plays an essential role in ensuring high quality wireless communications. Targeting for higher Quality of Service (QoS), many existing network antenna tuning schemes are based on expert knowledge, rule-based policies or conventional optimization theory. However, maximizing the traffic-related QoS does not guarantee that all customers experience good services. In addition, existing schemes are often limited to some handcrafted rules or heuristics and lack of adaptability especially in a time-varying environment. Quality of Experience (QoE), a user-centric metric, can better measure users' satisfaction for services in wireless networks. This paper proposes the cooperative tuning of antennas based on QoE, a paradigm shift from network-centric QoS to user-centric QoE domain. In a normal cellular network, besides the need of improving the overall QoE, it requires handling faults from different cells. As Multi-agent Reinforcement Learning (MARL) has the capability of self-learning the dynamics of environment, we propose an antenna configuration algorithm based on multi-goal MARL. In our framework, agents from different cells not only need to cooperate with each other to achieve the global goal of increasing the overall QoE of the wireless network but also complete some personal goals by combating the faults encountered in their own cells. To accelerate the training efficiency, we introduce a novel two-stage curriculum learning. To reduce the collection time of each QoE sample, we develop an accurate and timely QoE/QoS mapping model with the cascading of a Random Forest Classifier (RFC) and a Deep Neural Network (DNN) (abbreviated as RFC-DNN), which can help us obtain QoE by collecting QoS measurements and perform QoE-based antenna configurations with smaller time granularity. Our proposed RFC-DNN model can reduce the time by 70% when predicting the QoE of a single sample. A huge amount of time will be saved in MARL when tens of thousands of transitions/samples need to be collected. The performance results show that our proposed antenna tuning schemes can not only address specific faults in each cell, but also significantly improve the global average QoE with a faster and more stable convergence speed.

Low Risk Antenna Configurations for Mobile Communication Systems: A Safe Reinforcement Learning Method

Safe Exploration in Wireless Security: A Safe Reinforcement Learning Algorithm With Hierarchical Structure

Automated Antenna Design via Domain Knowledge-Informed Reinforcement Learning and Imitation Learning

Scalable Antenna Orientation Optimization for mmWave Mobile Communication Systems

Multi-Agent Reinforcement Learning with Common Policy for Antenna Tilt Optimization

An Antenna Optimization Framework Based on Deep Reinforcement Learning

Safe RAN control: A Symbolic Reinforcement Learning Approach

QoE-driven Antenna Tuning in Cellular Networks With Cooperative Multi-agent Reinforcement Learning

Model Based Residual Policy Learning with Applications to Antenna Control

Safe Multi-Agent Reinforcement Learning for Wireless Applications Against Adversarial Communications

Deep Reinforcement Learning Based on Location-Aware Imitation Environment for RIS-Aided Mmwave MIMO Systems

Pervasive Machine Learning for Smart Radio Environments Enabled by Reconfigurable Intelligent Surfaces

RIS-Aided Proactive Mobile Network Downlink Interference Suppression: A Deep Reinforcement Learning Approach

Safe-NORA: Safe Reinforcement Learning-based Mobile Network Resource Allocation for Diverse User Demands

Reinforcement Learning Based Antenna Selection in User-Centric Massive MIMO

Intelligent Reflecting Surface Configurations for Smart Radio Using Deep Reinforcement Learning

Deep Reinforcement Learning Evolution Algorithm for Dynamic Antenna Control in Multi-Cell Configuration HAPS System

Multi-agent Reinforcement Learning with Graph Q-Networks for Antenna Tuning

Adaptive Modulation Scheme for Satellite Communication Channel Based on RLNN

Cognitive Conformal Antenna Array Exploiting Deep Reinforcement Learning Method