Abstract:With the exponential growth of mobile users, ensuring high-quality network coverage has become paramount. Large-scale mobile networks consist of numerous base stations (BSs), each with adjustable parameters such as angles and beam widths. Automatically optimizing network coverage can be difficult due to environmental factors and the interdependence of the adjustable parameters. Due to the inherent uncertainties and unpredictable nature of large-scale wireless networks, traditional methods such as heuristics and meta-heuristics lack the adaptability and scalability required to cope with their dynamic environment. To address these challenges, we propose utilizing digital twin and reinforcement learning (RL) techniques within mobile networks characterized by multiple collaborating agents. We initially introduce DT-SimNet, a digital twin-enabled mobile network simulator to facilitate optimization evaluation. DT-SimNet can efficiently simulate communication behaviors of network elements within a complex environment while revealing user mobility patterns. Moreover, to address challenges arising from multifaceted relationships among users, BSs, and the parameters across BSs, we introduce an innovative strategy named Optimized Multi-Agent Proximal Policy Optimization with Self-supervised Prediction (OMAPPO-SSP). Compared to MAPPO, which leads to limited applicability and inferior performance due to the dynamic characteristics of 5G networks, this approach leverages network structure optimization and a self-supervised prediction mechanism, employing multi-agent reinforcement learning (MARL) principles to enhance efficiency. By harnessing collaborative neural networks, OMAPPO-SSP facilitates the explicit learning of behavioral interactions among all BSs, enabling effective decision-making in environments characterized by intricate spatial relationships, dynamic user behaviors, and diverse interactions. Extensive experiments are conducted to validate the efficiency and effectiveness of the OMAPPO-SSP. Within the target area, OMAPPO-SSP achieves a coverage ratio of 94.66% and an average throughput of 89746 bits per second (bps), demonstrating significant improvements compared to competing methods.

Mobility Robustness Optimization for Self-Organizing Networks: A Deep Reinforcement Learning Approach

Self-Organizing Mobility Robustness Optimization in LTE Networks with eICIC

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Joint Server Selection, Cooperative Offloading and Handover in Multi-access Edge Computing Wireless Network: A Deep Reinforcement Learning Approach

Mobility Robustness Optimization In Femtocell Networks Based On Ant Colony Algorithm

A novel handover scheme for millimeter wave network: An approach of integrating reinforcement learning and optimization

Deep Reinforcement Learning-Based Offloading Decision Optimization in Mobile Edge Computing

Deep-Mobility: A Deep Learning Approach for an Efficient and Reliable 5G Handover

A Deep Reinforcement Learning-based Approach for Adaptive Handover Protocols in Mobile Networks

Mobility-Aware Service Migration for Seamless Provision: A Reinforcement Learning Approach

Coverage Optimization for Large-Scale Mobile Networks with Digital Twin and Multi-Agent Reinforcement Learning

Deep Q-Learning for Self-Organizing Networks Fault Management and Radio Performance Improvement

Efficient Drone Mobility Support Using Reinforcement Learning

Conflict Avoidance between Mobility Robustness Optimization and Mobility Load Balancing

Decentralized Computation Offloading for Multi-User Mobile Edge Computing: A Deep Reinforcement Learning Approach

Mobility-Aware Resource Allocation in IoRT Network for Post-Disaster Communications with Parameterized Reinforcement Learning

Deep reinforcement learning for RAN optimization and control

Mobility robustness optimization in self-organizing LTE femtocell networks

Lyapunov-Guided Deep Reinforcement Learning for Stable Online Computation Offloading in Mobile-Edge Computing Networks

Mobility Management for Cellular-Connected UAVs: Model Based Versus Learning Based Approaches for Service Availability

Enhancing Handover for 5G mmWave Mobile Networks Using Jump Markov Linear System and Deep Reinforcement Learning