Abstract:Traditional Federated Learning (FL) is a promising paradigm that enables massive edge clients to collaboratively train deep neural network (DNN) models without exposing raw data to the parameter server (PS). To avoid the bottleneck on the PS, Decentralized Federated Learning (DFL), which utilizes peer-to-peer (P2P) communication without maintaining a global model, has been proposed. Nevertheless, DFL still faces two critical challenges, i.e., limited communication bandwidth and not independent and identically distributed (non-IID) local data, thus hindering efficient model training. Existing works commonly assume full model aggregation at periodic intervals, i.e., clients periodically collect models from peers. To reduce the communication cost, these methods allow clients to collect model(s) from selected peers, but often result in a significant degradation of model accuracy when dealing with non-IID data. Alternatively, the layer-wise aggregation mechanism has been proposed to alleviate communication overhead under the PS architecture, but its potential in DFL remains rarely explored yet. To this end, we propose an efficient DFL framework YOGA that adaptively performs layer-wise model aggregation and training. Specifically, YOGA first generates the ranking of layers in the model according to the learning speed and layer-wise divergence. Combining with the layer ranking and peers’ status information (i.e., data distribution and communication capability), we propose the max-match (MM) algorithm to generate the proper layer-wise model aggregation policy for the clients. Extensive experiments on DNN models and datasets show that YOGA saves communication cost by about 45% without sacrificing the model performance compared with the baselines, and provides 1.53-3.5 $\times$ speedup on the physical platform.

Layer-wise Adaptive Model Aggregation for Scalable Federated Learning

AsyncFedED: Asynchronous Federated Learning with Euclidean Distance Based Adaptive Weight Aggregation

FedPA: An adaptively partial model aggregation strategy in Federated Learning

Communication-Efficient Federated Deep Learning With Layerwise Asynchronous Model Update and Temporally Weighted Aggregation

Communication-Efficient Model Aggregation with Layer Divergence Feedback in Federated Learning

Layer-wise and Dimension-wise Locally Adaptive Federated Learning

Is Aggregation the Only Choice? Federated Learning via Layer-wise Model Recombination

Communication-Efficient Federated Deep Learning with Asynchronous Model Update and Temporally Weighted Aggregation

FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation

YOGA: Adaptive Layer-Wise Model Aggregation for Decentralized Federated Learning

FedSL: A Communication Efficient Federated Learning with Split Layer Aggregation

FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning

Enhancing Edge-Assisted Federated Learning with Asynchronous Aggregation and Cluster Pairing

A hierarchical federated learning model with adaptive model parameter aggregation

Adaptive Clustering-Based Model Aggregation for Federated Learning with Imbalanced Data

Federated Learning with Flexible Architectures

FedAgg: Adaptive Federated Learning with Aggregated Gradients

Federated Submodel Averaging.

A High-Performance Federated Learning Aggregation Algorithm Based on Learning Rate Adjustment and Client Sampling

Efficient Federated Learning Using Layer-Wise Regulation and Momentum Aggregation*

Agglomerative Federated Learning: Empowering Larger Model Training via End-Edge-Cloud Collaboration