Abstract:In mobile edge computing (MEC), randomly offloading tasks to edge servers (ES) can cause wireless devices (WD) to compete for limited bandwidth resources, leading to overall performance degradation. Reinforcement learning can provide suitable strategies for task offloading and resource allocation through exploration and trial-and-error, helping to avoid blind offloading. However, traditional reinforcement learning algorithms suffer from slow convergence and a tendency to get stuck in suboptimal local minima, significantly impacting the energy consumption and data timeliness of edge computing task unloading. To address these issues, we propose Parallel Exploration with Asynchronous Training-based Deep Reinforcement Learning (PEATDRL) algorithm for MEC network offloading decisions. Its objective is to maximize system performance while limiting energy consumption in an MEC environment characterized by time-varying wireless channels and random user task arrivals. Firstly, our model employs two independent DNNs for parallel exploration, each generating different offloading strategies. This parallel exploration enhances environmental adaptability, avoids the limitations of a single DNN, and addresses the issue of agents getting stuck in suboptimal local minima due to the explosion of decision combinations, thereby improving decision performance. Secondly, we set different learning rates for the two DNNs during the training phase and trained them at various intervals. This asynchronous training strategy increases the randomness of decision exploration, prevents the two DNNs from converging to the same suboptimal local solution, and improves convergence efficiency by enhancing sample utilization. Finally, we examine the impact of different parallel levels and training step differences on system performance metrics and explain the parameter choices. Experimental results show that the proposed method provides a viable solution to the performance issues caused by slow convergence and local minima, with PEATDRL improving task queue convergence speed by more than 20% compared to baseline algorithms.

Decentralized Task Offloading in Edge Computing: An Offline-to-Online Reinforcement Learning Approach

Edge Collaborative Online Task Offloading Method Based on Reinforcement Learning

An Offline-Transfer-Online Framework for Cloud-Edge Collaborative Distributed Reinforcement Learning

A Meta Reinforcement Learning-Based Task Offloading Strategy for IoT Devices in an Edge Cloud Computing Environment

Edge QoE: Computation Offloading With Deep Reinforcement Learning for Internet of Things

FLIRRAS: Fast Learning With Integrated Reward and Reduced Action Space for Online Multitask Offloading

Dependent Task Offloading for Edge Computing based on Deep Reinforcement Learning

Lyapunov-Guided Deep Reinforcement Learning for Stable Online Computation Offloading in Mobile-Edge Computing Networks

A3C-DO: A Regional Resource Scheduling Framework Based on Deep Reinforcement Learning in Edge Scenario

Deep Reinforcement Learning Method for Task Offloading in Mobile Edge Computing Networks Based on Parallel Exploration with Asynchronous Training

Dynamic Task Offloading in Edge Computing based on Dependency-aware Reinforcement Learning

Dependent Task Offloading in Edge Computing Using GNN and Deep Reinforcement Learning

QoE-Based Task Offloading With Deep Reinforcement Learning in Edge-Enabled Internet of Vehicles

A2C-DRL: Dynamic Scheduling for Stochastic Edge-Cloud Environments Using A2C and Deep Reinforcement Learning

An Efficient Online Computation Offloading Approach for Large-Scale Mobile Edge Computing via Deep Reinforcement Learning

Decentralized Computation Offloading for Multi-User Mobile Edge Computing: A Deep Reinforcement Learning Approach

Deep reinforcement learning-based online task offloading in mobile edge computing networks

Real-Time Offloading for Dependent and Parallel Tasks in Cloud-Edge Environments Using Deep Reinforcement Learning

Deep Reinforcement Learning Empowers Wireless Powered Mobile Edge Computing: Towards Energy-Aware Online Offloading

Offline Retraining for Online RL: Decoupled Policy Learning to Mitigate Exploration Bias

Deep Reinforcement Learning-Based Offloading Decision Optimization in Mobile Edge Computing