Abstract:To distribute the storage and computation load caused by growing capacity of deep neural network (DNN), collaborative intelligence (CI) framework has been proposed, where a deep model is split and executed in two distributed devices respectively. Intermediate feature must be transferred from the front end to the back in order to perform distributed inference, thus transmission process is the bottleneck that influences the inference efficiency in terms of accuracy and delay. Specifically for a bandwidth-limited human-in-loop visual analysis task, feature compression approach needs exploration to reduce the data volume to be transmitted, in order to achieve low transmission delay as well as maintain analysis performance and human perception ability. In this paper, the redundancy of intermediate feature both in spatial and statistical levels are firstly analyzed. A mathematical expression for the goal of feature compression is formulated, based on which a two-level redundancy removal based low-rate feature compression approach is proposed. For the front-end device, an information squeezing (IS) module is developed to squeeze the key information of input image and inject them into a low-resolution image. Then a backbone network is split into two parts with respects to the application demands of CI, and can be deployed at the front and back ends correspondingly. With a specifically designed objective function, IS module and the partitioned backbone network are optimized collaboratively to reduce the two-level redundancy, thus compressing the intermediate feature. A generative adversarial network (GAN)-based restoration module is proposed to recover an image with original resolution from the compressed feature, for satisfying human perception. Comprehensive experiments are conduct to validate the efficiency of the proposed method.

RCIF: Towards Robust Distributed DNN Collaborative Inference under Highly Lossy Networks

Low-Rate Feature Compression for Collaborative Intelligence: Reducing Redundancy in Spatial and Statistical Levels

Collaborative Inference for Deep Neural Networks in Edge Environments

Failure-Resilient Distributed Inference with Model Compression over Heterogeneous Edge Devices

A Fine-Grained End-to-End Latency Optimization Framework for Wireless Collaborative Inference

Reliable adaptive edge-cloud collaborative DNN inference acceleration scheme combining computing and communication resources in optical networks

A DNN partitioning framework with controlled lossy mechanisms for edge-cloud collaborative intelligence

Collaborative Edge AI Inference over Cloud-RAN

AC-DNN: An Adaptive Compact DNNs Architecture for Collaborative Learning Among Heterogeneous Smart Devices

End-Edge Collaborative Inference of Convolutional Fuzzy Neural Networks for Big Data-Driven Internet of Things

Adaptive Early Exiting for Collaborative Inference over Noisy Wireless Channels

RCDM: Enabling Robustness for Conditional Diffusion Model

Grad-FEC: Unequal Loss Protection of Deep Features in Collaborative Intelligence

Improving Device-Edge Cooperative Inference of Deep Learning via 2-Step Pruning

Learning Dynamic Computing Resource Allocation in Convolutional Neural Networks for Wireless Interference Identification

An Adaptive DNN Inference Acceleration Framework with End–edge–cloud Collaborative Computing

Toward Decentralized and Collaborative Deep Learning Inference for Intelligent IoT Devices

IEACC: an Intelligent Edge-Aided Congestion Control Scheme for Named Data Networking with Deep Reinforcement Learning.

Dynamic DNN Decomposition for Lossless Synergistic Inference

RL-DistPrivacy: Privacy-Aware Distributed Deep Inference for low latency IoT systems

SecoInfer: Secure DNN End-Edge Collaborative Inference Framework Optimizing Privacy and Latency