Abstract:This paper studies federated learning for nonparametric regression in the context of distributed samples across different servers, each adhering to distinct differential privacy constraints. The setting we consider is heterogeneous, encompassing both varying sample sizes and differential privacy constraints across servers. Within this framework, both global and pointwise estimation are considered, and optimal rates of convergence over the Besov spaces are established. Distributed privacy-preserving estimators are proposed and their risk properties are investigated. Matching minimax lower bounds, up to a logarithmic factor, are established for both global and pointwise estimation. Together, these findings shed light on the tradeoff between statistical accuracy and privacy preservation. In particular, we characterize the compromise not only in terms of the privacy budget but also concerning the loss incurred by distributing data within the privacy framework as a whole. This insight captures the folklore wisdom that it is easier to retain privacy in larger samples, and explores the differences between pointwise and global estimation under distributed privacy constraints.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to achieve optimal joint learning in non - parametric regression in a distributed sample environment, combined with heterogeneous differential privacy constraints of different servers. Specifically, the paper focuses on: 1. **Establishing the optimal convergence rate**: Under the premise of satisfying the Differential Privacy (DP) constraint, estimate the optimal convergence rate of the non - parametric regression function. This includes both global estimation and point - by - point estimation cases, and these convergence rates are defined in Besov spaces. 2. **Constructing the optimal estimator**: Propose a distributed privacy - protected estimator and study its risk properties. By matching the minimax lower bound (up to a logarithmic factor), prove the optimality of the proposed estimator in global estimation and point - by - point estimation. 3. **Quantifying the trade - off between statistical accuracy and privacy protection**: Explore the trade - off relationship between statistical accuracy and privacy protection under different privacy budget conditions. In particular, in the case where it is easier to maintain privacy in larger samples, analyze the impact of distributed data on the overall privacy framework. ### Problem Background With the proliferation of personal data and the progress of technology, protecting privacy has become extremely important. Differential Privacy (DP), as a widely adopted privacy framework, ensures that the results of statistical analysis do not disclose any sensitive information. However, in practical applications, data are often distributed on multiple servers, and each server may have different privacy requirements. Therefore, how to conduct effective statistical estimation in such a distributed environment while ensuring privacy is an important research topic. ### Research Method The paper considers a heterogeneous distributed environment, where each server has a different number of samples and the privacy constraints of each server are also different. In this setting, the authors propose two different differential privacy estimators for global estimation and point - by - point estimation respectively. Through strict theoretical analysis, the authors derive the minimax convergence rates of these two estimators and prove their optimality. ### Main Contributions 1. **Quantifying the cost of differential privacy**: For the risks of global and point - by - point estimation, quantify the cost brought by differential privacy. In particular, when the privacy budget is small, the minimax estimation error will increase significantly. 2. **Revealing phenomena under heterogeneous privacy constraints**: In the case where servers have heterogeneous privacy budgets, some interesting phenomena different from the homogeneous setting are discovered. For example, greater smoothness will exacerbate the impact of privacy constraints on the estimation error. 3. **Providing theoretical basis**: Provide a theoretical basis for developing federated learning algorithms that balance distributed privacy and accuracy. Understanding the optimal convergence rate helps to design more efficient algorithms to deal with privacy challenges in practical applications. ### Conclusion This paper conducts an in - depth study of the optimal joint learning problem combined with heterogeneous differential privacy constraints in non - parametric regression. Through strict theoretical analysis and experimental verification, the authors not only establish the optimal convergence rate, but also reveal the impact of privacy constraints on statistical estimation. These findings provide valuable references for future research and help promote the development of privacy - protected machine learning.

Optimal Federated Learning for Nonparametric Regression with Heterogeneous Distributed Differential Privacy Constraints

Projected Federated Averaging with Heterogeneous Differential Privacy.

Federated Transfer Learning with Differential Privacy

Federated Nonparametric Hypothesis Testing with Differential Privacy Constraints: Optimal Rates and Adaptive Tests

Differentially Private Federated Learning: Servers Trustworthiness, Estimation, and Statistical Inference

Differentially Private Federated Learning on Heterogeneous Data

Minimax And Adaptive Transfer Learning for Nonparametric Classification under Distributed Differential Privacy Constraints

Federated Learning with Differential Privacy

Efficient federated learning privacy preservation method with heterogeneous differential privacy

Federated Learning with Bayesian Differential Privacy

Differentially Private Federated Learning with Laplacian Smoothing

Optimal Private and Communication Constraint Distributed Goodness-of-Fit Testing for Discrete Distributions in the Large Sample Regime

A Non-parametric View of FedAvg and FedProx: Beyond Stationary Points

FedSeC: a Robust Differential Private Federated Learning Framework in Heterogeneous Networks

Towards Robust Federated Analytics via Differentially Private Measurements of Statistical Heterogeneity

Privacy, accuracy, and model fairness trade-offs in federated learning

The Impact of Differential Privacy on Model Fairness in Federated Learning.

Federated PCA and Estimation for Spiked Covariance Matrices: Optimal Rates and Efficient Algorithm

Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learning

Private Heterogeneous Federated Learning Without a Trusted Server Revisited: Error-Optimal and Communication-Efficient Algorithms for Convex Losses

Preserving Differential Privacy Between Features in Distributed Estimation