Abstract:Antenna tuning plays an essential role in ensuring high quality wireless communications. Targeting for higher Quality of Service (QoS), many existing network antenna tuning schemes are based on expert knowledge, rule-based policies or conventional optimization theory. However, maximizing the traffic-related QoS does not guarantee that all customers experience good services. In addition, existing schemes are often limited to some handcrafted rules or heuristics and lack of adaptability especially in a time-varying environment. Quality of Experience (QoE), a user-centric metric, can better measure users' satisfaction for services in wireless networks. This paper proposes the cooperative tuning of antennas based on QoE, a paradigm shift from network-centric QoS to user-centric QoE domain. In a normal cellular network, besides the need of improving the overall QoE, it requires handling faults from different cells. As Multi-agent Reinforcement Learning (MARL) has the capability of self-learning the dynamics of environment, we propose an antenna configuration algorithm based on multi-goal MARL. In our framework, agents from different cells not only need to cooperate with each other to achieve the global goal of increasing the overall QoE of the wireless network but also complete some personal goals by combating the faults encountered in their own cells. To accelerate the training efficiency, we introduce a novel two-stage curriculum learning. To reduce the collection time of each QoE sample, we develop an accurate and timely QoE/QoS mapping model with the cascading of a Random Forest Classifier (RFC) and a Deep Neural Network (DNN) (abbreviated as RFC-DNN), which can help us obtain QoE by collecting QoS measurements and perform QoE-based antenna configurations with smaller time granularity. Our proposed RFC-DNN model can reduce the time by 70% when predicting the QoE of a single sample. A huge amount of time will be saved in MARL when tens of thousands of transitions/samples need to be collected. The performance results show that our proposed antenna tuning schemes can not only address specific faults in each cell, but also significantly improve the global average QoE with a faster and more stable convergence speed.

An Antenna Optimization Framework Based on Deep Reinforcement Learning

Automated Antenna Design via Domain Knowledge-Informed Reinforcement Learning and Imitation Learning

An Efficient Antenna Optimization Framework Based on Inverse Radial Basis Function Network

An Efficient Antenna Optimization Framework Based on the NP to P Problem

Optimizing Wireless Coverage and Capacity with PPO-Based Adaptive Antenna Configuration

A Machine Learning Generative Method for Automating Antenna Design and Optimization

Reinforcement Learning for Antenna Selection and Optimization of Irregular Reconfigurable Intelligent Surfaces

A Compensation for Elevated Sidelobe of Radiation Pattern of Antenna Array Caused by Amplitude and Phase Discretization Based on Deep Reinforcement Learning

Multi-Antenna Tuning Simulation Platform by Deep Reinforcement Learning

Cognitive Conformal Antenna Array Exploiting Deep Reinforcement Learning Method

Hybrid Differential Evolution with Artificial Bee Colony and Its Application for Design of a Reconfigurable Antenna Array with Discrete Phase Shifters

Deep neural learning based optimization for automated high performance antenna designs

Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications

QoE-driven Antenna Tuning in Cellular Networks With Cooperative Multi-agent Reinforcement Learning

Bandwidth Improvement for Patch Antenna Via Knowledge-Based Deep Reinforcement Learning

Low Risk Antenna Configurations for Mobile Communication Systems: A Safe Reinforcement Learning Method

A Deep Learning Framework for Evaluating the Over-the-Air Performance of the Antenna in Mobile Terminals

Antenna Optimization Based on Auto-Context Broad Learning System

Active Reconfigurable Intelligent Surface-Assisted Mainlobe Wideband RFI Mitigation With Deep Reinforcement Learning for a Large Reflector Antenna

Joint Task Offloading and Resource Allocation for Intelligent Reflecting Surface-Aided Integrated Sensing and Communication Systems Using Deep Reinforcement Learning Algorithm