Abstract:Federated learning (FL) has been used as a promising approach to breaking the dilemma between the data privacy and the learning from large collections of distributed data. Without sharing data, the server trains a global model with high accuracy only by accessing the gradient of each client. However, recent research has revealed that FL is vulnerable to gradient inversion attacks, where the server can reconstruct clients' training data by inverting their uploaded gradients. Current defense methods share the common limitation that they need to either perturb the training data or the gradients. However, this perturbation will inevitably reduce the accuracy of the global model. To overcome these limitations, this paper proposes a defense method by allowing clients to train local models without using training data but rather using the statistical information of the data. Inspired by the statistical machine unlearning method, which uses statistical information to train models, combined with knowledge distillation to lightweight local models, our method further isolates the relationship between gradients and training data by transferring the model to a simpler model. Knowledge distillation typically involves extracting knowledge from a teacher model and transferring it to a student model without the need for the original training data. With this method, on the one hand, the server can reconstruct only the statistical information of the student model data, which is semantically meaningless to humans. On the other hand, using the statistical information of the data to train the global model can guarantee the model's accuracy. Experimental results show that our method outperforms existing defense methods in various aspects, including reconstruction accuracy, model accuracy and training efficiency.

KDRSFL: A knowledge distillation resistance transfer framework for defending model inversion attacks in split federated learning

ResSFL: A Resistance Transfer Framework for Defending Model Inversion Attack in Split Federated Learning

HYDRA-FL: Hybrid Knowledge Distillation for Robust and Accurate Federated Learning

Defending against gradient inversion attacks in federated learning via statistical machine unlearning

Defense Against Reconstruction Attacks in Split Federated Learning Through Decreasing Correlation Between Inputs and Activations.

Personalized and privacy-enhanced federated learning framework via knowledge distillation

An Empirical Study of the Inherent Resistance of Knowledge Distillation Based Federated Learning to Targeted Poisoning Attacks

Efficient Federated Learning for AIoT Applications Using Knowledge Distillation

KDk: A Defense Mechanism Against Label Inference Attacks in Vertical Federated Learning

Provable Defense against Privacy Leakage in Federated Learning from Representation Perspective

FLIP: A Provable Defense Framework for Backdoor Mitigation in Federated Learning

Data Distribution Inference Attack in Federated Learning Via Reinforcement Learning Support

FDSFL: Filtering Defense Strategies Toward Targeted Poisoning Attacks in IIoT-Based Federated Learning Networking System.

Federated Split Learning Via Mutual Knowledge Distillation

DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learning

Towards Understanding Adversarial Transferability in Federated Learning

Model Extraction Attacks on Split Federated Learning

KoReA-SFL: Knowledge Replay-based Split Federated Learning Against Catastrophic Forgetting

SCFL: Mitigating Backdoor Attacks in Federated Learning Based on SVD and Clustering.

Digital Twin-Assisted Knowledge Distillation Framework for Heterogeneous Federated Learning