HE2C: A Holistic Approach for Allocating Latency-Sensitive AI Tasks across Edge-Cloud

Minseo Kim,Wei Shu,Mohsen Amini Salehi

2024-11-29

Abstract:The high computational, memory, and energy demands of Deep Learning (DL) applications often exceed the capabilities of battery-powered edge devices, creating difficulties in meeting task deadlines and accuracy requirements. Unlike previous solutions that optimize a single metric (e.g., accuracy or energy efficiency), HE2C framework is designed to holistically address the latency, memory, accuracy, throughput, and energy demands of DL applications across edge-cloud continuum, thereby, delivering a more comprehensive and effective user experience. HE2C comprises three key modules: (a) a "feasibility-check module that evaluates the likelihood of meeting deadlines across both edge and cloud resources; (b) a "resource allocation strategy" that maximizes energy efficiency without sacrificing the inference accuracy; and (c) a "rescue module" that enhances throughput by leveraging approximate computing to trade accuracy for latency when necessary. Our primary objective is to maximize system prolong battery lifespan, throughput, and accuracy while adhering to strict latency constraints. Experimental evaluations in the context of wearable technologies for blind and visually impaired users demonstrate that HE2C significantly improves task throughput via completing a larger number of tasks within their specified deadlines, while preserving edge device battery and maintaining prediction accuracy with minimal latency impact. These results underscore HE2C's potential as a robust solution for resource management in latency-sensitive, energy-constrained edge-to-cloud environments.

Distributed, Parallel, and Cluster Computing

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are as follows: When deep learning (DL) applications run on edge devices, due to high computing, memory and energy requirements, they often exceed the capabilities of these battery - powered edge devices, making it difficult to meet task deadline and accuracy requirements. Specifically:

1. **Resource limitations of edge devices**: Edge devices usually have limited computing and memory resources. This makes running multiple neural network tasks simultaneously and continuously may lead to resource exhaustion, thus limiting task throughput and may lead to a high task - drop rate, affecting the user experience.

2. **Strict latency constraints**: Many deep - learning applications, especially wearable technologies designed for blind and visually - impaired users, need to operate under strict latency constraints to ensure real - time and responsiveness.

3. **Trade - off between energy efficiency and task accuracy**: When allocating resources in the edge - cloud environment, it is necessary to balance energy efficiency and task accuracy to ensure that the system can maximize battery life, throughput and accuracy while meeting latency requirements.

To solve these problems, the paper proposes a framework named HE2C, which aims to comprehensively optimize resource allocation from the edge to the cloud, considering multiple factors such as latency, memory, accuracy, throughput and energy requirements, thereby providing a more comprehensive and effective user experience. The HE2C framework contains three key modules:

- **Feasibility check module**: Evaluate the possibility of completing tasks on edge and cloud resources.
- **Resource allocation strategy**: Maximize energy efficiency without sacrificing inference accuracy.
- **Rescue module**: Improve throughput through approximate calculation and trade accuracy for latency when necessary.

The experimental results show that HE2C significantly improves task throughput, extends the battery life of edge devices, and maintains prediction accuracy while minimizing the impact of latency. These results emphasize the potential of HE2C as a robust solution for resource management in the edge - to - cloud environment.