Abstract:With the interdisciplinary advances of mobile communication and edge computing, massive heterogeneous tasks are accessing wireless networks and competing for the edge-end computing and communication resources. Digital twin (DT), which establishes the digital models of physical objects for simulation, analysis and optimization, provides a promising method for network scheduling and management. This paper proposes a DT-driven edge-end collaborative scheduling algorithm for heterogeneous tasks and heterogeneous computing/communication resources. Specifically, multiple end devices (EDs) cooperate with each other to accomplish a complex job, where each ED can offload individual task to multiple edge servers (ESs) for parallel computing. By fully considering deadline requirements of heterogeneous tasks, maximum computing capabilities of ESs and EDs, computing resource estimation deviations of DT, maximum transmit powers of EDs and tolerable peak interference powers to coexisting EDs, we formulate a job completion time minimization problem to jointly optimize the edge-end task division, transmit power control, computing resource type matching and allocation. To solve this non-convex problem, we first reformulate it by multi-agent Markov decision process, where a compound reward leveraging latency reward and deadline reward according to the task criticality is designed. Then, we propose a multi-agent deep reinforcement learning-based scheduling algorithm, where Actor-Critic framework with estimation and target networks is designed for policy and value iterations. Meanwhile, a step-by-step $\epsilon $ -greedy algorithm is proposed to balance exploration and exploitation, avoiding local optimal trap. Through offline centralized training by DT and online distributed execution by EDs, we realize edge-end collaborative computing for heterogeneous tasks. Experimental results demonstrate that, comparing with typical benchmark algorithms, the proposed algorithm converges with the highest reward and achieves the smallest job completion time, where the deadlines of heterogeneous tasks can be well satisfied respectively.

Joint Task and Computing Resource Allocation in Distributed Edge Computing Systems Via Multi-Agent Deep Reinforcement Learning

Stable Resource Allocation Based on Multi-agent Reinforcement Learning for Edge Computing

Load Balancing for Task Scheduling Based on Multi-Agent Reinforcement Learning in Cloud-Edge-End Collaborative Environments

A Task Offloading and Resource Allocation Strategy Based on Multi-Agent Reinforcement Learning in Mobile Edge Computing

Multi-agent Reinforcement Learning for Task Offloading with Hybrid Decision Space in Multi-Access Edge Computing

Digital Twin-Driven Collaborative Scheduling for Heterogeneous Task and Edge-End Resource Via Multi-Agent Deep Reinforcement Learning.

Joint Data Offloading and Resource Allocation for Multi-Cloud Heterogeneous Mobile Edge Computing Using Multi-Agent Reinforcement Learning

Distributed Task Offloading Based on Multi-Agent Deep Reinforcement Learning.

Multi-Agent Deep Reinforcement Learning-Based Interdependent Computing for Mobile Edge Computing-Assisted Robot Teams

Multi-agent Reinforcement Learning for Resource Allocation in IoT networks with Edge Computing

MADDPG-based joint optimization of task partitioning and computation resource allocation in mobile edge computing

Cooperative Task Offloading for Mobile Edge Computing Based on Multi-Agent Deep Reinforcement Learning

Distributed Multi-Cloud Multi-Access Edge Computing by Multi-Agent Reinforcement Learning

Edge Dynamic Service Offloading Based on Multi-agent Deep Q Learning.

Two-Timescale Joint Optimization of Task Scheduling and Resource Scaling in Multi-Data Center System Based on Multi-Agent Deep Reinforcement Learning

Reinforcement Learning for Task Placement in Collaborative Cloud- Edge Computing

Multi-Agent Reinforcement Learning for Distributed Joint Communication and Computing Resource Allocation over Cell-Free Massive MIMO-enabled Mobile Edge Computing Network

Multi-Agent Deep Reinforcement Learning Offloading Algorithm Based on Resource Reservation and Task Adjustment

Joint Job Offloading and Resource Allocation for Distributed Deep Learning in Edge Computing.

Joint computation offloading and resource allocation for end-edge collaboration in internet of vehicles via multi-agent reinforcement learning

Joint Computation Offloading and Service Caching in Mobile Edge-Cloud Computing Via Deep Reinforcement Learning