Abstract:With the interdisciplinary advances of mobile communication and edge computing, massive heterogeneous tasks are accessing wireless networks and competing for the edge-end computing and communication resources. Digital twin (DT), which establishes the digital models of physical objects for simulation, analysis and optimization, provides a promising method for network scheduling and management. This paper proposes a DT-driven edge-end collaborative scheduling algorithm for heterogeneous tasks and heterogeneous computing/communication resources. Specifically, multiple end devices (EDs) cooperate with each other to accomplish a complex job, where each ED can offload individual task to multiple edge servers (ESs) for parallel computing. By fully considering deadline requirements of heterogeneous tasks, maximum computing capabilities of ESs and EDs, computing resource estimation deviations of DT, maximum transmit powers of EDs and tolerable peak interference powers to coexisting EDs, we formulate a job completion time minimization problem to jointly optimize the edge-end task division, transmit power control, computing resource type matching and allocation. To solve this non-convex problem, we first reformulate it by multi-agent Markov decision process, where a compound reward leveraging latency reward and deadline reward according to the task criticality is designed. Then, we propose a multi-agent deep reinforcement learning-based scheduling algorithm, where Actor-Critic framework with estimation and target networks is designed for policy and value iterations. Meanwhile, a step-by-step $\epsilon $ -greedy algorithm is proposed to balance exploration and exploitation, avoiding local optimal trap. Through offline centralized training by DT and online distributed execution by EDs, we realize edge-end collaborative computing for heterogeneous tasks. Experimental results demonstrate that, comparing with typical benchmark algorithms, the proposed algorithm converges with the highest reward and achieves the smallest job completion time, where the deadlines of heterogeneous tasks can be well satisfied respectively.

Edge-edge Collaboration Based Micro-service Deployment in Edge Computing Networks

Microservice Auto-Scaling Algorithm Based on Workload Prediction in Cloud-Edge Collaboration Environment

Joint Optimization of Microservice Deployment and Routing in Edge Via Multi-Objective Deep Reinforcement Learning

Distributed Orchestration of Service Function Chains for Edge Intelligence in the Industrial Internet of Things

Graph Neural Network Aided Deep Reinforcement Learning for Microservice Deployment in Cooperative Edge Computing

Adaptive Edge Service Deployment in Burst Load Scenarios Using Deep Reinforcement Learning

Edge Cloud Selection Strategy for Micro-service Chain Migration in Mobile Edge Computing

Dynamic Microservice Deployment and Offloading for Things-Edge-Cloud Computing

A Collaborative Service Deployment and Application Assignment Method for Regional Edge Computing Enabled IoT

IoT Microservice Deployment in Edge-Cloud Hybrid Environment Using Reinforcement Learning

Microservice Selection in Edge-Cloud Collaborative Environment: A Deep Reinforcement Learning Approach

Service Placement Strategy for Joint Network Selection and Resource Scheduling in Edge Computing

Digital Twin-assisted Reinforcement Learning for Resource-aware Microservice Offloading in Edge Computing

Deep Reinforcement Learning Based Task Scheduling in Edge Computing Networks.

Digital Twin-Driven Collaborative Scheduling for Heterogeneous Task and Edge-End Resource Via Multi-Agent Deep Reinforcement Learning.

Deep Reinforcement Learning based Approach for Online Service Placement and Computation Resource Allocation in Edge Computing

A Kubernetes-Oriented Edge Network Orchestrator for Heterogeneous Environment

Transfer Learning Based Algorithm for Service Deployment Under Microservice Architecture

Intelligent Resource Allocation for Edge-Cloud Collaborative Networks: A Hybrid DDPG-D3QN Approach

Deep Reinforcement Learning for Joint Service Placement and Request Scheduling in Mobile Edge Computing Networks.

Deep Reinforcement Learning-Based Workload Scheduling for Edge Computing