Abstract:Federated learning (FL) emerged as a paradigm designed to improve data privacy by enabling data to reside at its source, thus embedding privacy as a core consideration in FL architectures, whether centralized or decentralized. Contrasting with recent findings by Pasquini et al., which suggest that decentralized FL does not empirically offer any additional privacy or security benefits over centralized models, our study provides compelling evidence to the contrary. We demonstrate that decentralized FL, when deploying distributed optimization, provides enhanced privacy protection - both theoretically and empirically - compared to centralized approaches. The challenge of quantifying privacy loss through iterative processes has traditionally constrained the theoretical exploration of FL protocols. We overcome this by conducting a pioneering in-depth information-theoretical privacy analysis for both frameworks. Our analysis, considering both eavesdropping and passive adversary models, successfully establishes bounds on privacy leakage. We show information theoretically that the privacy loss in decentralized FL is upper bounded by the loss in centralized FL. Compared to the centralized case where local gradients of individual participants are directly revealed, a key distinction of optimization-based decentralized FL is that the relevant information includes differences of local gradients over successive iterations and the aggregated sum of different nodes' gradients over the network. This information complicates the adversary's attempt to infer private data. To bridge our theoretical insights with practical applications, we present detailed case studies involving logistic regression and deep neural networks. These examples demonstrate that while privacy leakage remains comparable in simpler models, complex models like deep neural networks exhibit lower privacy risks under decentralized FL.

Distributed Modelling Approaches for Data Privacy Preserving

Decentralized Iot Data Sharing: A Blockchain-Based Federated Learning Approach with Joint Optimizations for Efficiency and Privacy

From distributed machine learning to federated learning: In the view of data privacy and security

Training Encrypted Models with Privacy-preserved Data on Blockchain

Privacy preserving distributed machine learning with federated learning

Efficient Privacy-Preserving Machine Learning in Hierarchical Distributed System

Federated Extra-Trees with Privacy Preserving

Privacy-Preserving and Reliable Distributed Federated Learning

Provable Privacy Advantages of Decentralized Federated Learning via Distributed Optimization

Decentralized Federated Learning Preserves Model and Data Privacy

A Study of Federated Learning with Internet of Things for Data Privacy and Security using Privacy Preserving Techniques

Research on Privacy Protection in Federated Learning Combining Distillation Defense and Blockchain

A Practical Privacy-preserving Method in Federated Deep Learning

Privacy-preserving Decentralized Federated Deep Learning.

Towards blockchain-enabled decentralized and secure federated learning

A Distributed Approach to Enabling Privacy-Preserving Model-Based Classifier Training

Distributed Intelligent Model for Privacy and Secrecy in Preschool Education

Privacy-Preserving Machine Learning Algorithms for Big Data Systems

Dynamic Secure Multi Broad Network for Privacy Preserving of Streaming Data

Distributed Private Online Learning for Social Big Data Computing over Data Center Networks

Preserving Model Privacy for Machine Learning in Distributed Systems.