Abstract:International Journal of Software Engineering and Knowledge Engineering, Ahead of Print. In mobile edge computing environment, intelligent inference services driven by DNN are highly sensitive to latency. Recently, collaborative inference between User Devices and Edge Servers (ESs) based on Deep Neural Networks (DNN) partition has achieved success in service acceleration. However, most of the existing collaborative acceleration schemes are partitioned for a single DNN inference task, which cannot quickly make partition decisions for a set of concurrent inference tasks, and often sacrifice inference accuracy. In addition, due to the limited resources of ESs, there is resource competition among concurrent requests, which makes the partitioned tasks cannot be offloaded to ESs in time for processing. Therefore, designing an efficient offloading scheme becomes essential. The task offloading schemes based on deep reinforcement learning can solve complex decision-making problems in high-dimensional state space, but they have problems such as insufficient sample diversity and easily falling into local optimum. In this paper, a Collaborative Inference Acceleration Scheme integrating DNN Partitioning and Task Offloading (CIAS-PnO) is proposed. First, while ensuring inference accuracy, the Collaborative DNN Layer Partitioning (CDLP) algorithm is designed with the goal of optimal latency. CDLP can reduce the problem scale of concurrent inference tasks partition by pruning operation and determine the partition decisions in time. Then, the Distributed Soft Actor-Critic (SAC)-based Partition Task Offloading algorithm (DSACO) is designed. DSACO supports SAC Agents to explore samples in parallel and share learning experiences, and uses the automatic entropy adjustment mechanism to improve the exploration efficiency of Agents, so as to avoid falling into local optimum and achieve efficient offloading of partition tasks. Experimental results on DNN benchmarks show that compared with the baseline acceleration schemes, CIAS-PnO achieves more than 19.8% acceleration performance improvement, and has higher convergence performance and task success rate.

Collaborative Inference for MEC Services Based on Multimodal Deep Neural Network.

Energy-Efficient Collaborative Inference in MEC: A Multi-Agent Reinforcement Learning Based Approach

Reinforcement Learning Based Energy-Efficient Collaborative Inference for Mobile Edge Computing

A Deep Reinforcement Learning Approach for DNN Collaborative Inference in Mobile Edge Computing

Multi-Agent Collaborative Inference via DNN Decoupling: Intermediate Feature Compression and Edge Learning

Multi-Agent Collaborative Inference Via DNN Decoupling: Intermediate Feature Compression and Edge Learning

Multi-Agent Systems for Collaborative Inference Based on Deep Policy Q-Inference Network

Collaborative Deep Neural Network Inference via Mobile Edge Computing

An Intelligent Collaborative Inference Approach of Service Partitioning and Task Offloading for Deep Learning Based Service in Mobile Edge Computing Networks.

Collaborative DNNs Inference with Joint Model Partition and Compression in Mobile Edge-Cloud Computing Networks

Learning-Based Edge-Device Collaborative DNN Inference in IoVT Networks

Collaborative Inference for Deep Neural Networks in Edge Environments

Distributed DNN Inference with Fine-grained Model Partitioning in Mobile Edge Computing Networks

DNN Real-Time Collaborative Inference Acceleration with Mobile Edge Computing

A Multi-Head Ensemble Multi-Task Learning Approach for Dynamical Computation Offloading

MoEI: Mobility-Aware Edge Inference Based on Model Partition and Service Migration

Collaborative Inference Acceleration Integrating DNN Partitioning and Task Offloading in Mobile Edge Computing

Joint DNN partitioning and task offloading in mobile edge computing via deep reinforcement learning

Joint Optimization of DNN Partition and Continuous Task Scheduling for Digital Twin-Aided MEC Network With Deep Reinforcement Learning

CCIED: Cache-aided Collaborative Intelligence Between Edge Devices.

Accelerating Deep Neural Network Tasks Through Edge-Device Adaptive Inference