Abstract:Two-party split learning has emerged as a popular paradigm for vertical federated learning. To preserve the privacy of the label owner, split learning utilizes a split model, which only requires the exchange of intermediate representations (IRs) based on the inputs and gradients for each IR between two parties during the learning process. However, split learning has recently been proven to survive label inference attacks. Though several defense methods could be adopted, they either have limited defensive performance or significantly negatively impact the original mission. In this paper, we propose a novel two-party split learning method to defend against existing label inference attacks while maintaining the high utility of the learned models. Specifically, we first craft a dimension transformation module, SecDT, which could achieve bidirectional mapping between original labels and increased K-class labels to mitigate label leakage from the directional perspective. Then, a gradient normalization algorithm is designed to remove the magnitude divergence of gradients from different classes. We propose a softmax-normalized Gaussian noise to mitigate privacy leakage and make our K unknowable to adversaries. We conducted experiments on real-world datasets, including two binary-classification datasets (Avazu and Criteo) and three multi-classification datasets (MNIST, FashionMNIST, CIFAR-10); we also considered current attack schemes, including direction, norm, spectral, and model completion attacks. The detailed experiments demonstrate our proposed method's effectiveness and superiority over existing approaches. For instance, on the Avazu dataset, the attack AUC of evaluated four prominent attacks could be reduced by 0.4532+-0.0127.

Shuffled Transformers for Blind Training

VILLAIN: Backdoor Attacks Against Vertical Split Learning

Privacy-Preserving Vision Transformer on Permutation-Encrypted Images

Privacy-Preserving Split Learning with Vision Transformers using Patch-Wise Random and Noisy CutMix

Split Ways: Privacy-Preserving Training of Encrypted Data Using Split Learning

I can't see it but I can Fine-tune it: On Encrypted Fine-tuning of Transformers using Fully Homomorphic Encryption

Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer

Weights Shuffling for Improving DPSGD in Transformer-based Models

LayerShuffle: Enhancing Robustness in Vision Transformers by Randomizing Layer Execution Order

Protecting Split Learning by Potential Energy Loss

Human-imperceptible, Machine-recognizable Images

Seeing the Forest through the Trees: Data Leakage from Partial Transformer Gradients

APRIL: Finding the Achilles' Heel on Privacy for Vision Transformers

Love or Hate? Share or Split? Privacy-Preserving Training Using Split Learning and Homomorphic Encryption

Boosting Adversarial Transferability by Block Shuffle and Rotation

PPTIF: Privacy-Preserving Transformer Inference Framework for Language Translation

Delving into Differentially Private Transformer

Secure Split Learning Against Property Inference, Data Reconstruction, and Feature Space Hijacking Attacks

Training on Fake Labels: Mitigating Label Leakage in Split Learning via Secure Dimension Transformation

Dullahan: Stealthy Backdoor Attack against Without-Label-Sharing Split Learning

Transpose Attack: Stealing Datasets with Bidirectional Training