Abstract:In recent years, deep learning models have become ubiquitous in industry and academia alike. Deep neural networks can solve some of the most complex pattern-recognition problems today, but come with the price of massive compute and memory requirements. This makes the problem of deploying such large-scale neural networks challenging in resource-constrained mobile edge computing platforms, specifically in mission-critical domains like surveillance and healthcare. To solve this, a promising solution is to split resource-hungry neural networks into lightweight disjoint smaller components for pipelined distributed processing. At present, there are two main approaches to do this: semantic and layer-wise splitting. The former partitions a neural network into parallel disjoint models that produce a part of the result, whereas the latter partitions into sequential models that produce intermediate results. However, there is no intelligent algorithm that decides which splitting strategy to use and places such modular splits to edge nodes for optimal performance. To combat this, this work proposes a novel AI-driven online policy, SplitPlace, that uses Multi-Armed-Bandits to intelligently decide between layer and semantic splitting strategies based on the input task's service deadline demands. SplitPlace places such neural network split fragments on mobile edge devices using decision-aware reinforcement learning for efficient and scalable computing. Moreover, SplitPlace fine-tunes its placement engine to adapt to volatile environments. Our experiments on physical mobile-edge environments with real-world workloads show that SplitPlace can significantly improve the state-of-the-art in terms of average response time, deadline violation rate, inference accuracy, and total reward by up to 46, 69, 3 and 12 percent respectively.

Learning-Based Multi-Tier Split Computing for Efficient Convergence of Communication and Computation

Efficient Communication-Computation Tradeoff for Split Computing: A Multi-Tier Deep Reinforcement Learning Approach

Optimum splitting computing for DNN training through next generation smart networks: a multi-tier deep reinforcement learning approach

Efficient Partitioning and Communication Scheme-Based Distributed Edge Computing to Accelerate Deep Neural Network

Split Learning in Wireless Networks: A Communication and Computation Adaptive Scheme

Dynamic Split Computing for Efficient Deep EDGE Intelligence

Deep Reinforcement Learning for Edge Computing and Resource Allocation in 5G Beyond

Deep reinforcement learning‐based multitask hybrid computing offloading for multiaccess edge computing

Online Learning for Orchestration of Inference in Multi-User End-Edge-Cloud Networks

Joint Computation Offloading And Resource Configuration In Ultra-Dense Edge Computing Networks: A Deep Reinforcement Learning Solution

Two-Phase Split Computing Framework in Edge-Cloud Continuum

DynaSplit: A Hardware-Software Co-Design Framework for Energy-Aware Inference on Edge

Edge Intelligence for Energy-efficient Computation Offloading and Resource Allocation in 5G Beyond

Dynamic Encoding and Decoding of Information for Split Learning in Mobile-Edge Computing: Leveraging Information Bottleneck Theory

SplitPlace: AI Augmented Splitting and Placement of Large-Scale Neural Networks in Mobile Edge Environments

Enabling Edge Artificial Intelligence via Goal-oriented Deep Neural Network Splitting

Joint DNN partitioning and task offloading in mobile edge computing via deep reinforcement learning

Distributed and Collective Deep Reinforcement Learning for Computation Offloading: A Practical Perspective.

Accelerating Split Federated Learning over Wireless Communication Networks

Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges