Abstract:We propose a new approach for privacy-preserving and verifiable convolutional neural network (CNN) testing in a distrustful multi-stakeholder environment. The approach is aimed to enable that a CNN model developer convinces a user of the truthful CNN performance over non-public data from multiple testers, while respecting model and data privacy. To balance the security and efficiency issues, we appropriately integrate three tools with the CNN testing, including collaborative inference, homomorphic encryption (HE) and zero-knowledge succinct non-interactive argument of knowledge (zk-SNARK). We start with strategically partitioning a CNN model into a private part kept locally by the model developer, and a public part outsourced to an outside server. Then, the private part runs over the HE-protected test data sent by a tester, and transmits its outputs to the public part for accomplishing subsequent computations of the CNN testing. Second, the correctness of the above CNN testing is enforced by generating zk-SNARK based proofs, with an emphasis on optimizing proving overhead for two-dimensional (2-D) convolution operations, since the operations dominate the performance bottleneck during generating proofs. We specifically present a new quadratic matrix program (QMP)-based arithmetic circuit with a single multiplication gate for expressing 2-D convolution operations between multiple filters and inputs in a batch manner. Third, we aggregate multiple proofs with respect to a same CNN model but different testers' test data (i.e., different statements) into one proof, and ensure that the validity of the aggregated proof implies the validity of the original multiple proofs. Lastly, our experimental results demonstrate that our QMP-based zk-SNARK performs nearly 13.9x faster than the existing quadratic arithmetic program (QAP)-based zk-SNARK in proving time, and 17.6x faster in Setup time, for high-dimension matrix multiplication. Besides, the limitation on handling a bounded number of multiplications of QAP-based zk-SNARK is relieved.

CAPM: Fast and Robust Verification on Maxpool-based CNN via Dual Network

Towards General Robustness Verification of MaxPool-based Convolutional Neural Networks via Tightening Linear Approximation

MaxQ: Multi-Axis Query for N:M Sparsity Network

Channel Max Pooling For Image Classification

A improved pooling method for convolutional neural networks

Multiactivation Pooling Method in Convolutional Neural Networks for Image Recognition.

Certifying Robustness of Convolutional Neural Networks with Tight Linear Approximation

Fine-Grained Vehicle Classification with Channel Max Pooling Modified CNNs

Maximal Independent Vertex Set applied to Graph Pooling

PSRR-MaxpoolNMS: Pyramid Shifted MaxpoolNMS with Relationship Recovery

Mixed Pooling For Convolutional Neural Networks

Maximal Independent Sets for Pooling in Graph Neural Networks

Pvcnn: Privacy-Preserving and Verifiable Convolutional Neural Network Testing

A RRAM Based Max-Pooling Scheme for Convolutional Neural Network

zkCNN: Zero Knowledge Proofs for Convolutional Neural Network Predictions and Accuracy

Deep Generalized Max Pooling

An Overhead-Free Max-Pooling Method for SNN.

Group Reconstruction and Max-Pooling Residual Capsule Network

Deep Scattering Network with Max-pooling

Improving CNN Performance Accuracies with Min-Max Objective

Improving CNN Performance with Min-Max Objective