Abstract:Optimization of user association in a densely deployed heterogeneous cellular network is usually challenging and even more complicated due to the dynamic nature of user mobility and fluctuation in user counts. While deep reinforcement learning (DRL) emerges as a promising solution, its application in practice is hindered by high trial-and-error costs in real world and unsatisfactory physical network performance during training. In addition, existing DRL-based user association methods are usually only applicable to scenarios with a fixed number of users due to convergence and compatibility challenges. In this paper, we propose a parallel digital twin (DT)-driven DRL method for user association and load balancing in networks with both dynamic user counts, distribution, and mobility patterns. Our method employs a distributed DRL strategy to handle varying user numbers and exploits a refined neural network structure for faster convergence. To address these DRL training-related challenges, we devise a high-fidelity DT construction technique, featuring a zero-shot generative user mobility model, named Map2Traj, based on a diffusion model. Map2Traj estimates user trajectory patterns and spatial distributions solely from street maps. Armed with this DT environment, DRL agents are enabled to be trained without the need for interactions with the physical network. To enhance the generalization ability of DRL models for dynamic scenarios, a parallel DT framework is further established to alleviate strong correlation and non-stationarity in single-environment training and improve the training efficiency. Numerical results show that the proposed parallel DT-driven DRL method achieves closely comparable performance to real environment training, and even outperforms those trained in a single real-world environment with nearly 20% gain in terms of cell-edge user performance.

Learn to Augment Network Simulators Towards Digital Network Twins

Learnable Digital Twin for Efficient Wireless Network Evaluation

How AI-driven Digital Twins Can Empower Mobile Networks

Network Digital Twin: Context, Enabling Technologies and Opportunities

Digital Network Twins for Next-generation Wireless: Creation, Optimization, and Challenges

What-if Analysis Framework for Digital Twins in 6G Wireless Network Management

Distributed Deep Reinforcement Learning for Resource Allocation in Digital Twin Networks

Review and Assessment of Digital Twin--Oriented Social Network Simulators

Demo: Scalable Digital Twin System for Mobile Networks with Generative AI

Two-Timescale Digital Twin Assisted Model Interference and Retraining over Wireless Network

Twinning Commercial Network Traces on Experimental Open RAN Platforms

From Simulators to Digital Twins for Enabling Emerging Cellular Networks: A Tutorial and Survey

Digital Twin Empowered Heterogeneous Network Selection in Vehicular Networks With Knowledge Transfer

Mapping Wireless Networks into Digital Reality through Joint Vertical and Horizontal Learning

Generative AI-Driven Digital Twin for Mobile Networks

Parallel Digital Twin-driven Deep Reinforcement Learning for User Association and Load Balancing in Dynamic Wireless Networks

Learnable Wireless Digital Twins: Reconstructing Electromagnetic Field with Neural Representations

Digital Twin for Networking: A Data-driven Performance Modeling Perspective

Generative AI-enabled Digital Twins for 6G-enhanced Smart Cities

Sim2real for Reinforcement Learning Driven Next Generation Networks

DTRAN: A Special Use Case of RAN Optimization using Digital Twin