Abstract:High-quality private machine learning (ML) data stored in local data centers becomes a key competitive factor for AI corporations. In this paper, we present a novel insider attack called Matryoshka to reveal the possibility of breaking the privacy of ML data even with no exposed interface. Our attack employs a scheduled-to-publish DNN model as a carrier model for covert transmission of secret models which memorize the information of private ML data that otherwise has no interface to the outsider. At the core of our attack, we present a novel parameter sharing approach which exploits the learning capacity of the carrier model for information hiding. Our approach simultaneously achieves: (i) High Capacity - With almost no utility loss of the carrier model, Matryoshka can transmit over 10,000 real-world data samples within a carrier model which has 220× less parameters than the total size of the stolen data, and simultaneously transmit multiple heterogeneous datasets or models within a single carrier model under a trivial distortion rate, neither of which can be done with existing steganography techniques; (ii) Decoding Efficiency - once downloading the published carrier model, an outside colluder can exclusively decode the hidden models from the carrier model with only several integer secrets and the knowledge of the hidden model architecture; (iii) Effectiveness - Moreover, almost all the recovered models either have similar performance as if it is trained independently on the private data, or can be further used to extract memorized raw training data with low error; (iv) Robustness - Information redundancy is naturally implemented to achieve resilience against common post-processing techniques on the carrier before its publishing; (v) Covertness - A model inspector with different levels of prior knowledge could hardly differentiate a carrier model from a normal model.

Modeling Deep Learning Based Privacy Attacks on Physical Mail

The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks

Matryoshka: Exploiting the Over-Parametrization of Deep Learning Models for Covert Data Transmission

Backdoor Attacks Against Deep Learning Systems in the Physical World

Rethinking Privacy Preserving Deep Learning: How to Evaluate and Thwart Privacy Attacks

Data Stealing Attacks against Large Language Models via Backdooring

Teach LLMs to Phish: Stealing Private Information from Language Models

Physical Invisible Backdoor Based on Camera Imaging

Privacy Backdoors: Stealing Data with Corrupted Pretrained Models

A Brief Survey on Deep Learning Based Data Hiding

Privacy Analysis of Deep Learning in the Wild: Membership Inference Attacks against Transfer Learning

I Know What You See: Power Side-Channel Attack on Convolutional Neural Network Accelerators

Only My Model On My Data: A Privacy Preserving Approach Protecting one Model and Deceiving Unauthorized Black-Box Models

Protecting Privacy in Shared Photos Via Adversarial Examples Based Stealth

Hidden Data Privacy Breaches in Federated Learning

Matryoshka: Stealing Functionality of Private ML Data by Hiding Models in Model

Deep Model Intellectual Property Protection Via Deep Watermarking

Leaking Sensitive Financial Accounting Data in Plain Sight using Deep Autoencoder Neural Networks

Privacy-Preserving Eye-tracking Using Deep Learning

Privacy attacks against deep learning models and their countermeasures

RFLA: A Stealthy Reflected Light Adversarial Attack in the Physical World