Abstract:Antenna tuning plays an essential role in ensuring high quality wireless communications. Targeting for higher Quality of Service (QoS), many existing network antenna tuning schemes are based on expert knowledge, rule-based policies or conventional optimization theory. However, maximizing the traffic-related QoS does not guarantee that all customers experience good services. In addition, existing schemes are often limited to some handcrafted rules or heuristics and lack of adaptability especially in a time-varying environment. Quality of Experience (QoE), a user-centric metric, can better measure users' satisfaction for services in wireless networks. This paper proposes the cooperative tuning of antennas based on QoE, a paradigm shift from network-centric QoS to user-centric QoE domain. In a normal cellular network, besides the need of improving the overall QoE, it requires handling faults from different cells. As Multi-agent Reinforcement Learning (MARL) has the capability of self-learning the dynamics of environment, we propose an antenna configuration algorithm based on multi-goal MARL. In our framework, agents from different cells not only need to cooperate with each other to achieve the global goal of increasing the overall QoE of the wireless network but also complete some personal goals by combating the faults encountered in their own cells. To accelerate the training efficiency, we introduce a novel two-stage curriculum learning. To reduce the collection time of each QoE sample, we develop an accurate and timely QoE/QoS mapping model with the cascading of a Random Forest Classifier (RFC) and a Deep Neural Network (DNN) (abbreviated as RFC-DNN), which can help us obtain QoE by collecting QoS measurements and perform QoE-based antenna configurations with smaller time granularity. Our proposed RFC-DNN model can reduce the time by 70% when predicting the QoE of a single sample. A huge amount of time will be saved in MARL when tens of thousands of transitions/samples need to be collected. The performance results show that our proposed antenna tuning schemes can not only address specific faults in each cell, but also significantly improve the global average QoE with a faster and more stable convergence speed.

Optimizing Wireless Coverage and Capacity with PPO-Based Adaptive Antenna Configuration

DDPG with Transfer Learning and Meta Learning Framework for Resource Allocation in Underlay Cognitive Radio Network

Real-Time 3D MIMO Antenna Tuning with Deep Reinforcement Learning

Multi-Agent Reinforcement Learning with Common Policy for Antenna Tilt Optimization

An Antenna Optimization Framework Based on Deep Reinforcement Learning

Deep Reinforcement Learning-Assisted Optimization for Resource Allocation in Downlink OFDMA Cooperative Systems

Coverage Optimization for Large-Scale Mobile Networks with Digital Twin and Multi-Agent Reinforcement Learning

QoE-driven Antenna Tuning in Cellular Networks With Cooperative Multi-agent Reinforcement Learning

Low-Coupling Policy Optimization Framework for Power Allocation in Ultra-Dense Small-Cell Networks

Optimization-driven Deep Reinforcement Learning for Robust Beamforming in IRS-assisted Wireless Communications

Learning Deterministic Policy with Target for Power Control in Wireless Networks

Downlink Power Control for Cell-Free Massive MIMO with Deep Reinforcement Learning

Channel assignment and power allocation for throughput improvement with PPO in B5G heterogeneous edge networks

Scalable Antenna Orientation Optimization for mmWave Mobile Communication Systems

No-Pain No-Gain: DRL Assisted Optimization in Energy-Constrained CR-NOMA Networks

Digital Twin Enhanced Multi-Agent Reinforcement Learning for Large-Scale Mobile Network Coverage Optimization

Deep Multi-Agent Reinforcement Learning for Resource Allocation in D2D Communication Underlaying Cellular Networks

Multi-Objective Optimization of Energy Saving and Throughput in Heterogeneous Networks Using Deep Reinforcement Learning

Coverage and Capacity Optimization in STAR-RISs Assisted Networks: A Machine Learning Approach

Intelligent flying-beamformer for hybrid mmWave systems: A deep reinforcement learning approach

Hierarchical Reinforcement Learning for AP Duplex Mode Optimization in Network-Assisted Full-Duplex Cell-Free Networks