Abstract:In mobile edge computing (MEC), randomly offloading tasks to edge servers (ES) can cause wireless devices (WD) to compete for limited bandwidth resources, leading to overall performance degradation. Reinforcement learning can provide suitable strategies for task offloading and resource allocation through exploration and trial-and-error, helping to avoid blind offloading. However, traditional reinforcement learning algorithms suffer from slow convergence and a tendency to get stuck in suboptimal local minima, significantly impacting the energy consumption and data timeliness of edge computing task unloading. To address these issues, we propose Parallel Exploration with Asynchronous Training-based Deep Reinforcement Learning (PEATDRL) algorithm for MEC network offloading decisions. Its objective is to maximize system performance while limiting energy consumption in an MEC environment characterized by time-varying wireless channels and random user task arrivals. Firstly, our model employs two independent DNNs for parallel exploration, each generating different offloading strategies. This parallel exploration enhances environmental adaptability, avoids the limitations of a single DNN, and addresses the issue of agents getting stuck in suboptimal local minima due to the explosion of decision combinations, thereby improving decision performance. Secondly, we set different learning rates for the two DNNs during the training phase and trained them at various intervals. This asynchronous training strategy increases the randomness of decision exploration, prevents the two DNNs from converging to the same suboptimal local solution, and improves convergence efficiency by enhancing sample utilization. Finally, we examine the impact of different parallel levels and training step differences on system performance metrics and explain the parameter choices. Experimental results show that the proposed method provides a viable solution to the performance issues caused by slow convergence and local minima, with PEATDRL improving task queue convergence speed by more than 20% compared to baseline algorithms.

Reinforcement Learning Based Wireless Augmented Reality on Mobile Edge Computing

Learning-Driven Algorithms for Responsive AR Offloading with Non-Deterministic Rewards in Metaverse-Enabled MEC

Online Learning Algorithms for Offloading Augmented Reality Requests with Uncertain Demands in MECs

An Edge-Computing Based Architecture for Mobile Augmented Reality

Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence

Play to Earn in the Metaverse with Mobile Edge Computing over Wireless Networks: A Deep Reinforcement Learning Approach

Edge intelligence computing for mobile augmented reality with deep reinforcement learning approach

Wireless Multi-User Interactive Virtual Reality in Metaverse with Edge-Device Collaborative Computing

Beyond the Edge: An Advanced Exploration of Reinforcement Learning for Mobile Edge Computing, its Applications, and Future Research Trajectories

A Deep Reinforcement Learning Strategy for MEC Enabled Virtual Reality in Telecommunication Networks

Location-Dependent Augmented Reality Services in Wireless Edge-Enabled Metaverse Systems

Learning-based Prediction, Rendering and Association Optimization for MEC-enabled Wireless Virtual Reality (VR) Network

Reinforcement Learning-based Mobile Edge Computing and Transmission Scheduling for Video Surveillance

Optimizing the Learning Performance in Mobile Augmented Reality Systems with CNN

Mobile Edge Computing and AI Enabled Web3 Metaverse over 6G Wireless Communications: A Deep Reinforcement Learning Approach

Age-Based Scheduling for Mobile Edge Computing: A Deep Reinforcement Learning Approach

Feeling of Presence Maximization: mmWave-Enabled Virtual Reality Meets Deep Reinforcement Learning

An Efficient Online Computation Offloading Approach for Large-Scale Mobile Edge Computing via Deep Reinforcement Learning

Deep Reinforcement Learning Method for Task Offloading in Mobile Edge Computing Networks Based on Parallel Exploration with Asynchronous Training

Modeling on Resource Allocation for Age-Sensitive Mobile Edge Computing Using Federated Multi-Agent Reinforcement Learning

Attention-based QoE-aware Digital Twin Empowered Edge Computing for Immersive Virtual Reality