Abstract:Bidirectional privacy-preservation federated learning is crucial as both local gradients and the global model may leak privacy. However, only a few works attempt to achieve it, and they often face challenges such as excessive communication and computational overheads, or significant degradation of model accuracy, which hinders their practical applications. In this paper, we design an efficient and high-accuracy bidirectional privacy-preserving scheme for federated learning to complete secure model training and secure aggregation. To efficiently achieve bidirectional privacy, we design an efficient and accuracy-lossless model perturbation method on the server side (called $\mathbf{MP\_Server}$) that can be combined with local differential privacy (LDP) to prevent clients from accessing the model, while ensuring that the local gradients obtained on the server side satisfy LDP. Furthermore, to ensure model accuracy, we customize a distributed differential privacy mechanism on the client side (called $\mathbf{DDP\_Client}$). When combined with $\mathbf{MP\_Server}$, it ensures LDP of the local gradients, while ensuring that the aggregated result matches the accuracy of central differential privacy (CDP). Extensive experiments demonstrate that our scheme significantly outperforms state-of-the-art bidirectional privacy-preservation baselines (SOTAs) in terms of computational cost, model accuracy, and defense ability against privacy attacks. Particularly, given target accuracy, the training time of SOTAs is approximately $200$ times, or even over $1000$ times, longer than that of our scheme. When the privacy budget is set relatively small, our scheme incurs less than $6\%$ accuracy loss compared to the privacy-ignoring method, while SOTAs suffer up to $20\%$ accuracy loss. Experimental results also show that the defense capability of our scheme outperforms than SOTAs.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the two - way privacy protection problem in Federated Learning (FL). Specifically, the paper aims to design an efficient and high - precision two - way privacy protection scheme to achieve secure model training and secure aggregation, ensuring that the privacy of local gradients and the global model is not leaked during the federated learning process.
### Background and Problem Statement
#### 1. Privacy Challenges in Federated Learning
Federated learning allows multiple decentralized clients to collaboratively train a global model under the coordination of a central server without exchanging local data. Although this method has advantages in protecting data privacy, it still faces the risk of privacy attacks. For example:
- **Leakage of Local Gradients**: Local gradients may contain sensitive information, and attackers can infer the details of training data through these gradients.
- **Leakage of the Global Model**: The global model is regarded as a valuable asset, and unauthorized access, including by the clients themselves, must be prevented.
Therefore, two - way privacy protection is crucial, which is to protect both the local gradients of each client and the global model on the server.
#### 2. Limitations of Existing Methods
Existing Privacy - Preserving Federated Learning (PPFL) schemes usually can only protect privacy in one direction, that is, either protect the local gradients of clients or prevent clients from accessing the global model. Only a few works attempt to combine Secure Multi - party Computation (SMC) technology and Differential Privacy (DP) to achieve two - way privacy protection, but these methods have the following problems:
- **High Computation and Communication Overhead**: The use of SMC technology leads to high computation and communication costs, making it difficult to be deployed in practical applications.
- **Decline in Model Precision**: Existing methods often lead to a significant decline in model precision while protecting privacy.
### Goals of the Paper
To overcome the above problems, this paper proposes an efficient two - way privacy protection scheme with the following specific goals:
1. **Two - way Privacy Protection**: Ensure that the server cannot access the local gradients of clients, and at the same time, clients cannot obtain the global model.
2. **High Performance**: Ensure that the efficiency and model precision of the scheme are as close as possible to those of federated learning methods without privacy protection (such as FedAvg), so as to be convenient for practical applications.
### Overview of the Solution
The paper proposes a scheme that combines the Model Perturbation Method (MPServer) and the Distributed Differential Privacy Mechanism (DDP Client). Specifically:
- **MPServer**: An efficient and accuracy - non - lossy model perturbation method is designed on the server side, which can be combined with Local Differential Privacy (LDP) to prevent clients from accessing the model and at the same time ensure that the local gradients obtained by the server side meet LDP.
- **DDP Client**: A distributed differential privacy mechanism is customized on the client side, which is used in combination with MPServer to ensure that the local gradients meet LDP and at the same time make the aggregation result match the precision of Centralized Differential Privacy (CDP).
In this way, this scheme not only achieves efficient two - way privacy protection but also maintains a relatively high model precision.
### Experimental Verification
The experimental results show that this scheme is significantly superior to existing methods in terms of computational efficiency, model precision, and the ability to defend against privacy attacks. In particular, given the target precision, the training time of this scheme is more than 200 times or even 1000 times faster than that of existing methods. When the privacy budget is small, the precision loss of this scheme is less than 6%, while the precision loss of existing methods can reach 20%.
In summary, through the innovative MPServer and DDP Client methods, this paper successfully solves the two - way privacy protection problem in federated learning and demonstrates its high efficiency and superior performance in practical applications.