Abstract:Federated learning (FL) enables collaborative model training over distributed computing nodes without sharing their privacy-sensitive raw data. However, in FL, iterative exchanges of model updates between distributed nodes and the cloud server can result in significant communication cost, especially when the data distributions at distributed nodes are imbalanced with requiring more rounds of iterations. In this paper, with our in-depth empirical studies, we disclose that extensive cloud aggregations can be avoided without compromising the learning accuracy if frequent aggregations can be enabled at edge network. To this end, we shed light on the hierarchical federated learning (HFL) framework, where a subset of distributed nodes can play as edge aggregators to support edge aggregations. Under the HFL framework, we formulate a communication cost minimization (CCM) problem to minimize the total communication cost required for model learning with a target accuracy by making decisions on edge aggragator selection and node-edge associations. Inspired by our data-driven insights that the potential of HFL lies in the data distribution at edge aggregators, we propose ShapeFL, i.e., SHaping dAta distRibution at Edge, to transform and solve the CCM problem. In ShapeFL, we divide the original problem into two sub-problems to minimize the per-round communication cost and maximize the data distribution diversity of edge aggregator data, respectively, and devise two light-weight algorithms to solve them accordingly. Extensive experiments are carried out based on several opened datasets and real-world network topologies, and the results demonstrate the efficacy of ShapeFL in terms of both learning accuracy and communication efficiency.

Communication Efficient Federated Learning With Heterogeneous Structured Client Models

HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients

FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorization

Completely Heterogeneous Federated Learning

HDHRFL: A Hierarchical Robust Federated Learning Framework for Dual-Heterogeneous and Noisy Clients

Data-Quality-Driven Federated Learning for Optimizing Communication Costs

Privacy preserving federated learning for full heterogeneity

Communication-Efficient Federated Learning With Data and Client Heterogeneity

Improving Communication Efficiency: A Dual Adaptive Federated Learning Framework

Federated Learning via Indirect Server-Client Communications

A Communication-Efficient Hierarchical Federated Learning Framework via Shaping Data Distribution at Edge

Towards Efficient Federated Learning Framework Via Selective Aggregation of Models

Heterogeneous Federated Learning with Cross-layer Model Fusion.

Federated Learning with Additional Mechanisms on Clients to Reduce Communication Costs

Federated Learning Client Selection Mechanism under System and Data Heterogeneity

THF: 3-way Hierarchical Framework For Efficient Client Selection And Resource Management In Federated Learning

Communication-Efficient Model Aggregation with Layer Divergence Feedback in Federated Learning

FedHe: Heterogeneous Models and Communication-Efficient Federated Learning

Clustered Federated Learning in Heterogeneous Environment.

Dynamic Attention-based Communication-Efficient Federated Learning

Heterogeneity-Aware Federated Learning with Adaptive Client Selection and Gradient Compression.