Abstract:Supporting convolutional neural network (CNN) inference on resource-constrained IoT devices in a timely manner has been an outstanding challenge for emerging smart systems. To mitigate the burden on IoT devices, the prevailing solution is to offload the CNN inference task, which is usually composed of billions of operations, to public cloud. However, the "offloading-to-cloud" solution may cause privacy breach while moving sensitive data to cloud. For privacy protection, the research community has resorted to advanced cryptographic primitives and approximation techniques to support CNN inference on encrypted data. Consequently, these attempts cause impractical computational overhead on IoT devices and degrade the performance of CNNs. Moreover, relying on the remote cloud can cause additional network latency and even make the system dysfunction when network connection is off. We proposes an extremely lightweight edge device assisted private CNN inference solution for IoT devices, namely LEP-CNN. The main design of LEP-CNN is based on a novel online/offline encryption scheme. The decryption of LEP-CNN is pre-computed offline via utilizing the linear property of the most time-consuming operations of CNNs. As a result, LEP-CNN allows IoT devices to securely offload over 99% CNN operations, and edge devices to execute CNN inference on encrypted data as efficient as on plaintext. LEP-CNN also provides an integrity check option to help IoT devices detect error results with a successful rate over 99%. Experiments on AlexNet show that LEP-CNN can speed up the CNN inference for more than 35 times for resource constrained IoT devices. A homomorphic encryption based AlexNet using CryptoNets is implemented to compare with LEP-CNN to demonstrate that LEP-CNN has a better performance than homomorphic encryption based privacy preserving neural networks under time-sensitive scenarios.

Poster Abstract: CNN-guardian: Secure Neural Network Inference Acceleration on Edge GPU.

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

Privacy preserving Neural Network Inference on Encrypted Data with GPUs

ENIGMA: Low-Latency and Privacy-Preserving Edge Inference on Heterogeneous Neural Network Accelerators

Pratical Privacy-preserving Convolutional Neural Network Inference Framework with Edge Computing for Health Monitoring

GUARDIAN: A Hardware-Assisted Distributed Framework to Enhance Deep Learning Security

XNET: A Real-Time Unified Secure Inference Framework Using Homomorphic Encryption.

DistrEdge: Speeding up Convolutional Neural Network Inference on Distributed Edge Devices

LEP-CNN: A Lightweight Edge Device Assisted Privacy-preserving CNN Inference Solution for IoT

Towards the AlexNet Moment for Homomorphic Encryption: HCNN, theFirst Homomorphic CNN on Encrypted Data with GPUs

Efficient and Secure Deep Learning Inference in Trusted Processor Enabled Edge Clouds

cuSCNN: A Secure and Batch-Processing Framework for Privacy-Preserving Convolutional Neural Network Prediction on GPU

GFS-CNN: A GPU-friendly Secure Computation Platform for Convolutional Neural Networks

Toward Practical Privacy-Preserving Convolutional Neural Networks Exploiting Fully Homomorphic Encryption

Distributed Deep Learning Inference Acceleration using Seamless Collaboration in Edge Computing

Efficient Edge AI: Deploying Convolutional Neural Networks on FPGA with the Gemmini Accelerator

Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing

EdgeCI: Distributed Workload Assignment and Model Partitioning for CNN Inference on Edge Clusters

Adaptive Distributed Convolutional Neural Network Inference at the Network Edge with ADCNN

Design and Prototyping Distributed CNN Inference Acceleration in Edge Computing

A Unified Optimization Approach for CNN Model Inference on Integrated GPUs