Abstract:International Journal of Software Engineering and Knowledge Engineering, Ahead of Print. In mobile edge computing environment, intelligent inference services driven by DNN are highly sensitive to latency. Recently, collaborative inference between User Devices and Edge Servers (ESs) based on Deep Neural Networks (DNN) partition has achieved success in service acceleration. However, most of the existing collaborative acceleration schemes are partitioned for a single DNN inference task, which cannot quickly make partition decisions for a set of concurrent inference tasks, and often sacrifice inference accuracy. In addition, due to the limited resources of ESs, there is resource competition among concurrent requests, which makes the partitioned tasks cannot be offloaded to ESs in time for processing. Therefore, designing an efficient offloading scheme becomes essential. The task offloading schemes based on deep reinforcement learning can solve complex decision-making problems in high-dimensional state space, but they have problems such as insufficient sample diversity and easily falling into local optimum. In this paper, a Collaborative Inference Acceleration Scheme integrating DNN Partitioning and Task Offloading (CIAS-PnO) is proposed. First, while ensuring inference accuracy, the Collaborative DNN Layer Partitioning (CDLP) algorithm is designed with the goal of optimal latency. CDLP can reduce the problem scale of concurrent inference tasks partition by pruning operation and determine the partition decisions in time. Then, the Distributed Soft Actor-Critic (SAC)-based Partition Task Offloading algorithm (DSACO) is designed. DSACO supports SAC Agents to explore samples in parallel and share learning experiences, and uses the automatic entropy adjustment mechanism to improve the exploration efficiency of Agents, so as to avoid falling into local optimum and achieve efficient offloading of partition tasks. Experimental results on DNN benchmarks show that compared with the baseline acceleration schemes, CIAS-PnO achieves more than 19.8% acceleration performance improvement, and has higher convergence performance and task success rate.

Joint multi-user DNN partitioning and task offloading in mobile edge computing

Efficient Partitioning and Communication Scheme-Based Distributed Edge Computing to Accelerate Deep Neural Network

Deep Neural Network Task Partitioning and Offloading for Mobile Edge Computing

Task Partitioning and Offloading in DNN-Task Enabled Mobile Edge Computing Networks

Joint DNN partitioning and task offloading in mobile edge computing via deep reinforcement learning

Joint Optimization With DNN Partitioning and Resource Allocation in Mobile Edge Computing

Joint DNN Partition and Resource Allocation for Task Offloading in Edge-Cloud-Assisted IoT Environments

Joint Multi-User DNN Partitioning and Computational Resource Allocation for Collaborative Edge Intelligence

Energy-Efficient DNN Partitioning and Offloading for Task Completion Rate Maximization in Multiuser Edge Intelligence

Energy-Efficient Joint Partitioning and Offloading for Delay-Sensitive CNN Inference in Edge Computing

DNN Inference Task Offloading Based on Distributed Soft Actor-Critic in Mobile Edge Computing.

Joint Optimization of Model Partitioning and Resource Allocation for Edge Computing with Intermittently Operating Devices

Hastening Stream Offloading of Inference Via Multi-Exit DNNs in Mobile Edge Computing

Joint DNN partitioning and resource allocation for completion rate maximization of delay-aware DNN inference tasks in wireless powered mobile edge computing

Distributed DNN Inference with Fine-grained Model Partitioning in Mobile Edge Computing Networks

A DNN inference acceleration algorithm combining model partition and task allocation in heterogeneous edge computing system

HSMS-ADP: Adaptive DNNs Partitioning for End-Edge Collaborative Inference in High-Speed Mobile Scenarios

Joint Task Partition and Computation Offloading for Latency-Sensitive Services in Mobile Edge Networks

Towards Resource-aware DNN Partitioning for Edge Devices with Heterogeneous Resources

DNN Inference Acceleration with Partitioning and Early Exiting in Edge Computing

Collaborative Inference Acceleration Integrating DNN Partitioning and Task Offloading in Mobile Edge Computing