Abstract:End-edge collaborative inference enhances computational efficiency by segmenting a deep neural network (DNN) model into two parts, executed across the end device and the edge node. However, existing collaborative inference strategies often involve transmitting original inputs from the end device to the edge node, resulting in significant risks of user detail leakage without requiring input reconstruction. Therefore, in this work, we present SecoInfer, a secure layer-level DNN end-edge collaborative inference framework. SecoInfer achieves joint optimization of data privacy and inference latency for DNN partition solutions that meet latency constraints, supported by three key designs. First, the privacy-aware DNN layer projection measurement quantifies the difficulty adversaries encounter in reconstructing the original input from the intermediate output of each layer. Then, the latency-privacy integrated structure modelling enables the direct calculation of the privacy measurement and inference latency for each partition solution from a list element or a directed acyclic graph (DAG) cut. Finally, the two-stage latency constraint adjustment scheme narrows down the search space of feasible partition solutions at the block level and fine-tunes the final one to meet the latency constraint based on layer depth. We prototype SecoInfer, utilizing a Raspberry Pi 4B as the end device and a server with an NVIDIA GeForce RTX 3060 GPU as the edge node. Experimental results demonstrate that under latency constraints of 20 ms, 33 ms, and 40 ms, SecoInfer reduces adversarial data reconstruction by \(9.84\%\) , \(19.26\%\) , and \(25.18\%\) respectively, without any loss of task model accuracy. SecoInfer also enhances efficiency, reducing the time needed to determine optimal end-edge partition solutions on a Raspberry Pi 4B by \(18.04\%\) .

FastSecNet: An Efficient Cryptographic Framework for Private Neural Network Inference

CHEETAH: An Ultra-Fast, Approximation-Free, and Privacy-Preserved Neural Network Framework based on Joint Obscure Linear and Nonlinear Computations

GELU-Net: A Globally Encrypted, Locally Unencrypted Deep Neural Network for Privacy-Preserved Learning.

Industrial Scale Privacy Preserving Deep Neural Network

FssNN: Communication-Efficient Secure Neural Network Training via Function Secret Sharing

Faster CryptoNets: Leveraging Sparsity for Real-World Encrypted Inference

Communication-Efficient Privacy-Preserving Neural Network Inference via Arithmetic Secret Sharing

CrossNet: A Low-Latency MLaaS Framework for Privacy-Preserving Neural Network Inference on Resource-Limited Devices

PASNet: Polynomial Architecture Search Framework for Two-party Computation-based Secure Neural Network Deployment

PyHENet: A Generic Framework for Privacy-Preserving DL Inference Based on Fully Homomorphic Encryption

Towards Scalable and Privacy-Preserving Deep Neural Network via Algorithmic-Cryptographic Co-design

XNET: A Real-Time Unified Secure Inference Framework Using Homomorphic Encryption.

Towards Fast and Scalable Private Inference

SecoInfer: Secure DNN End-Edge Collaborative Inference Framework Optimizing Privacy and Latency

Toward Scalable and Privacy-preserving Deep Neural Network Via Algorithmic-Cryptographic Co-design

SecPE: Secure Prompt Ensembling for Private and Robust Large Language Models

Sphynx: ReLU-Efficient Network Design for Private Inference

PCNNCEC: Efficient and Privacy-Preserving Convolutional Neural Network Inference Based on Cloud-Edge-Client Collaboration

Bi-CryptoNets: Leveraging Different-Level Privacy for Encrypted Inference

Practical Two-party Privacy-preserving Neural Network Based on Secret Sharing

Crypto-Nets: Neural Networks over Encrypted Data