Abstract:Conventional machine learning (ML) and deep learning approaches require sharing customers' sensitive information with an external credit bureau to generate a prediction model, thereby increasing the risk of privacy leakage. This poses a significant challenge for financial companies. To address this challenge, federated learning has emerged as a promising approach to protect data privacy. However, the high communication costs associated with federated systems, particularly for large neural networks, can be a bottleneck. To mitigate this issue, it is necessary to limit the number and size of communications for practical training of large neural structures. Gradient sparsification is a technique that has gained increasing attention as a method to reduce communication costs, as it updates only significant gradients and accumulates insignificant gradients locally. However, the secure aggregation framework cannot directly employ gradient sparsification. To overcome this limitation, this article proposes two sparsification methods for reducing the communication costs of federated learning. The first method is a time-varying hierarchical sparsification method for model parameter updates, which addresses the challenge of maintaining model accuracy after a high sparsity ratio. This method can significantly reduce the cost of a single communication. The second method is to apply sparsification to the secure aggregation framework. Specifically, the encryption mask matrix is sparsified to reduce communication costs while protecting privacy. Experiments demonstrate that our method can reduce the upload communication costs to approximately 2.9% to 18.9% of the conventional federated learning algorithm under different non-IID experiment settings when the sparsity rate is 0.01.

Practical Lossless Federated Singular Vector Decomposition over Billion-Scale Data

Federated Singular Vector Decomposition.

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

Federated Learning for Sparse Principal Component Analysis

Efficient and Privacy-Preserving Feature Importance-based Vertical Federated Learning

Federated Learning for Privacy-Preserving Medical Data Analytics in Big Data

PVD-FL: A Privacy-Preserving and Verifiable Decentralized Federated Learning Framework

Communication-Efficient and Privacy-Preserving Large-Scale Federated Learning Counteracting Heterogeneity

Data Valuation for Vertical Federated Learning: A Model-free and Privacy-preserving Method

A Lightweight and Accuracy-Lossless Privacy-Preserving Method in Federated Learning

Accelerating Vertical Federated Learning

FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data

Federated Frank-Wolfe Algorithm

Efficient and Secure Federated Learning for Financial Applications

A Flexible Privacy-preserving Framework for Singular Value Decomposition under Internet of Things Environment

FedSel: Federated SGD Under Local Differential Privacy with Top-k Dimension Selection

Privacy-Preserving and Robust Federated Deep Metric Learning

Scalable and Privacy-Preserving Federated Principal Component Analysis

Differentially Private Federated Learning with Laplacian Smoothing

Federated Sufficient Dimension Reduction Through High-Dimensional Sparse Sliced Inverse Regression