Abstract:While federated learning (FL) eliminates the transmission of raw data over a network, it is still vulnerable to privacy breaches from the communicated model parameters. In this work, we propose Multi-Tier Federated Learning with Multi-Tier Differential Privacy (M^2FDP), a DP-enhanced FL methodology for jointly optimizing privacy and performance in hierarchical networks. One of the key concepts of M^2FDP is to extend the concept of HDP towards Multi-Tier Differential Privacy (MDP), while also adapting DP noise injection at different layers of an established FL hierarchy -- edge devices, edge servers, and cloud servers -- according to the trust models within particular subnetworks. We conduct a comprehensive analysis of the convergence behavior of M^2FDP, revealing conditions on parameter tuning under which the training process converges sublinearly to a finite stationarity gap that depends on the network hierarchy, trust model, and target privacy level. Subsequent numerical evaluations demonstrate that M^2FDP obtains substantial improvements in these metrics over baselines for different privacy budgets, and validate the impact of different system configurations.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: in a multi - tier federated learning (MFL) system, how to optimize the trade - off between privacy protection and model performance by introducing multi - tier differential privacy (MDP). Specifically, the research aims to: 1. **Expand the differential privacy mechanism**: Extend the traditional differential privacy (DP) mechanism from a single level to a multi - layer network structure to adapt to more complex hierarchical network environments. 2. **Optimize the balance between privacy and performance**: Inject different DP noises according to the trust model at different levels (edge devices, edge servers, and cloud servers) to ensure high model performance while protecting privacy. 3. **Analyze the convergence behavior**: Conduct a theoretical analysis of the convergence of the MFL system after introducing MDP, reveal the sub - linear convergence characteristics under parameter adjustment conditions, and explore the impact of DP noises at different levels on the convergence speed and model accuracy. ### Main contributions of the paper 1. **Formalize the M2FDP framework**: Propose a new method (M2FDP) that combines multi - tier federated learning and multi - tier differential privacy, which can implement a flexible trust model in a network with any number of layers and maintain the target privacy level throughout the training process. 2. **Convergence analysis**: Analyze the convergence behavior of M2FDP for non - convex machine learning loss functions, and prove that under an appropriate step - size selection, the cumulative average of the global model will converge sub - linearly at a rate of \( O\left(\frac{1}{\sqrt{T}}\right) \). 3. **Experimental verification**: Demonstrate significant improvements of M2FDP under different privacy budgets through numerical evaluation, and verify the impact of network configuration and trust model on training performance. ### Formula summary - **Definition of differential privacy**: \[ \text{For all adjacent datasets } D \text{ and } D' \text{, and all } S \subseteq \text{Range}(M), \text{ satisfy}: \] \[ \Pr[M(D) \in S] \leq e^\epsilon \Pr[M(D') \in S] + \delta \] - **Standard deviation of the Gaussian mechanism**: \[ \sigma > \sqrt{\frac{2 \log(1.25 / \delta)}{\epsilon}} \Delta f \] - **L2 sensitivity**: \[ \Delta f = \max_{D, D'} \| f(D) - f(D') \|_2 \] - **Noise variance**: \[ \sigma(\Delta_{l,c}) = \alpha_l q \Delta_{l,c} \sqrt{\frac{L \log(1 / \delta)}{\epsilon}} \] - **Convergence formula**: \[ \frac{1}{T} \sum_{t = 1}^T \| \nabla F(w(t)) \|^2 \leq 2\beta F(w(1)) \sqrt{T + 1} + K_{\max} \left( G^2 \left( 1 + \frac{1}{\beta} \right) + \sigma^2 \right) \frac{1}{T} \] \[ + 8LM (K_{\max})^4 q^2 \frac{\log(1 / \delta)}{\epsilon^2} \sum_{l = 1}^L (1 - p_{\min,l - 1})^2 \left( A_l + B_l + C_l \right) \] Through these contributions, this paper provides for privacy protection and performance optimization in multi - tier federated learning systems.

Differentially-Private Multi-Tier Federated Learning

Joint Communication and Learning Design of Differential Privacy for Federated Learning over Multi-cell Networks.

FDP-FL: Differentially Private Federated Learning with Flexible Privacy Budget Allocation

Differentially Private Federated Learning on Heterogeneous Data

Performance-Enhanced Federated Learning with Differential Privacy for Internet of Things

DMM: Distributed Matrix Mechanism for Differentially-Private Federated Learning using Packed Secret Sharing

Performance Analysis and Optimization in Privacy-Preserving Federated Learning

Differentially Private Federated Learning with an Adaptive Noise Mechanism

Enhancing Federated Learning with Adaptive Differential Privacy and Priority-Based Aggregation

Federated Learning with Differential Privacy

A Fine-Grained Differentially Private Federated Learning Against Leakage from Gradients

LDP-Fed+: A robust and privacy-preserving federated learning based classification framework enabled by local differential privacy

Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergence

Privacy-Preserving Federated Learning with Differentially Private Hyperdimensional Computing

Nikolai N. Anichkov and his theory of atherosclerosis.

Using Decentralized Aggregation for Federated Learning with Differential Privacy

Hierarchical federated learning with global differential privacy

Personalized Federated Learning With Differential Privacy and Convergence Guarantee

Privacy As a Resource in Differentially Private Federated Learning.

Convergent Differential Privacy Analysis for General Federated Learning: the $f$-DP Perspective

Adaptive Differential Privacy in Federated Learning: A Priority-Based Approach