Abstract:In recent years, deep learning (DL) models have demonstrated remarkable achievements on non-trivial tasks such as speech recognition, image processing, and natural language understanding. One of the significant contributors to the success of DL is the proliferation of end devices that act as a catalyst to provide data for data-hungry DL models. However, computing DL training and inference still remains the biggest challenge. Moreover, most of the time central cloud servers are used for such computation, thus opening up other significant challenges, such as high latency, increased communication costs, and privacy concerns. To mitigate these drawbacks, considerable efforts have been made to push the processing of DL models to edge servers (a mesh of computing devices near end devices). Recently, the confluence point of DL and edge has given rise to edge intelligence (EI), defined by the International Electrotechnical Commission (IEC) as the concept where the data is acquired, stored, and processed utilizing edge computing with DL and advanced networking capabilities. Broadly, EI has six levels of categories based on the three locations where the training and inference of DL take place, e.g., cloud server, edge server, and end devices. This survey paper focuses primarily on the fifth level of EI, called all in-edge level, where DL training and inference (deployment) are performed solely by edge servers. All in-edge is suitable when the end devices have low computing resources, e.g., Internet-of-Things, and other requirements such as latency and communication cost are important such as in mission-critical applications (e.g., health care). Besides, 5G/6G networks are envisioned to use all in-edge. Firstly, this paper presents all in-edge computing architectures, including centralized, decentralized, and distributed. Secondly, this paper presents enabling technologies, such as model parallelism, data parallelism, and split learning, which facilitates DL training and depl- yment at edge servers. Thirdly, model adaptation techniques based on model compression and conditional computation are described because the standard cloud-based DL deployment cannot be directly applied to all in-edge due to its limited computational resources. Fourthly, this paper discusses eleven key performance metrics to evaluate the performance of DL at all in-edge efficiently. Finally, several open research challenges in the area of all in-edge are presented.

An Empirical Study of Distributed Deep Learning Training on Edge (student Abstract)

Deep Learning in the Era of Edge Computing: Challenges and Opportunities

Efficient Partitioning and Communication Scheme-Based Distributed Edge Computing to Accelerate Deep Neural Network

A Distributed Privacy-Preserving Framework for Deep Learning with Edge-Cloud Computing.

Enabling Deep Learning on Edge Devices

Self-aware distributed deep learning framework for heterogeneous IoT edge devices

Edge-PRUNE: Flexible Distributed Deep Learning Inference

Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy

EdgeKE: An On-Demand Deep Learning IoT System for Cognitive Big Data on Industrial Edge Devices

Deep Learning With Edge Computing: A Review

Distributed Convolutional Neural Network Training on Mobile and Edge Clusters

DeepThings: Distributed Adaptive Deep Learning Inference on Resource-Constrained IoT Edge Clusters

Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing

Low Latency Deep Learning Inference Model for Distributed Intelligent IoT Edge Clusters

Enabling All In-Edge Deep Learning: A Literature Review

CoEdge: Cooperative DNN Inference With Adaptive Workload Partitioning Over Heterogeneous Edge Devices

Rethinking Machine Learning Development and Deployment for Edge Devices

Optimum splitting computing for DNN training through next generation smart networks: a multi-tier deep reinforcement learning approach

Distributed Training for Deep Learning Models On An Edge Computing Network Using ShieldedReinforcement Learning

Characterizing Resource Heterogeneity in Edge Devices for Deep Learning Inferences

Toward Collaborative Inferencing of Deep Neural Networks on Internet-of-Things Devices