Abstract:Federated learning for training models over mobile devices is gaining popularity. Current systems for this task exhibit significant trade-offs between model accuracy, privacy guarantee, and device efficiency. For instance, Oort (OSDI 2021) provides excellent accuracy and efficiency but requires a trusted central server. On the other hand, Orchard (OSDI 2020) provides good accuracy and the rigorous guarantee of differential privacy over an untrusted server, but creates huge overhead for the devices. This paper describes Aero, a new federated learning system that significantly improves this trade-off. Aero guarantees good accuracy, differential privacy over an untrusted server, and keeps the device overhead low. The key idea of Aero is to tune system architecture and design to a specific set of popular, federated learning algorithms. This tuning requires novel optimizations and techniques, e.g., a new protocol to securely aggregate updates from devices. An evaluation of Aero demonstrates that it provides comparable accuracy to plain federated learning (without differential privacy), and it improves efficiency (CPU and network) over Orchard by up to $10^5\times$.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to achieve a balance among high - precision, strict privacy protection and low device overhead in federated learning. Specifically: 1. **High - precision**: Conduct federated learning without sacrificing the precision of model training. 2. **Strict privacy protection**: Provide Differential Privacy (DP) guarantees, and protect the privacy of user data even in an untrusted server environment. 3. **Low device overhead**: Reduce the computing and network overhead of devices during the process of participating in federated learning, enabling them to operate efficiently. Existing federated learning systems have significant trade - offs among these three goals: - Systems such as **Oort** provide high - precision and efficient device performance, but require a trusted central server, which may not be feasible in practical applications. - **Orchard** provides good precision and strict Differential Privacy guarantees, but in an untrusted server environment, the device overhead is very large. To overcome these trade - offs, the paper proposes a new federated learning system **Aero**, and its main contributions are: - **High - precision**: Aero provides model precision comparable to ordinary federated learning (without Differential Privacy). - **Strict privacy protection**: Under the same threat model (i.e., an untrusted server), Aero provides Differential Privacy guarantees. - **Low device overhead**: Aero significantly reduces the computing and network overhead of devices, keeping the device overhead at a low level. The key idea of Aero is to optimize the system architecture and design for a specific class of federated learning algorithms (such as DP - FedAvg and DP - FedSGD), and achieve the above goals through the following technical means: - **Multi - level fine - grained sum tree**: Utilize the sampling characteristics of the DP - FedAvg algorithm, and share the verification work through multiple fine - grained sum trees, thereby reducing device overhead. - **Polynomial identity test**: Optimize the verification process of device sum tree nodes to further improve efficiency. - **Key reuse**: Reuse keys between different rounds to reduce the overhead of key generation while preventing malicious attacks. In conclusion, Aero successfully finds a balance among high - precision, strict privacy protection and low device overhead through a series of innovative technical means, providing a new solution for the practical application of federated learning.

Federated learning with differential privacy and an untrusted aggregator

Federated learning with differential privacy and an untrusted aggregator

Projected Federated Averaging with Heterogeneous Differential Privacy.

Over-the-air Learning Rate Optimization for Federated Learning

Hybrid Differential Privacy Based Federated Learning for Internet of Things

Performance-Enhanced Federated Learning with Differential Privacy for Internet of Things

Asynchronous Federated Learning with Differential Privacy for Edge Intelligence

Auditable Homomorphic-based Decentralized Collaborative AI with Attribute-based Differential Privacy

A Lightweight and Accuracy-Lossless Privacy-Preserving Method in Federated Learning

AeroRec: an Efficient On-Device Recommendation Framework Using Federated Self-Supervised Knowledge Distillation

Using Decentralized Aggregation for Federated Learning with Differential Privacy

The Impact of Differential Privacy on Model Fairness in Federated Learning.

HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning

Decentralized Wireless Federated Learning with Differential Privacy

Federated Transfer Learning with Differential Privacy

PAGroup: Privacy-Aware Grouping Framework for High-Performance Federated Learning

Privacy, accuracy, and model fairness trade-offs in federated learning

Mixed-Precision Federated Learning via Multi-Precision Over-The-Air Aggregation

An efficient privacy-preserving and verifiable scheme for federated learning

RR-LADP: A Privacy-Enhanced Federated Learning Scheme for Internet of Everything.

Privacy-Preserving Model Aggregation for Asynchronous Federated Learning