Abstract:Federated learning is a paradigm that proposes protecting data privacy by sharing local models instead of raw data during each iteration of model training. However, these models can be large, with many parameters, provoking a substantial communication cost and having a notable environmental impact. Reducing communication overhead is paramount but conflictual to maintaining the model's accuracy. Most research has dealt with the different factors influencing communication reduction separately without addressing their correlations. Moreover, most of them do not consider the heterogeneity of clients' hardware. Finding the optimal configuration to fulfil all these training aspects can become intractable for classical techniques. This work explores the add-in that multi-objective evolutionary algorithms can provide for solving the communication overhead problem while achieving high accuracy. We do this by 1) realistically modelling and formulating this task as a multi-objective problem by considering the devices' heterogeneity, 2) including all the communication-triggering aspects, and 3) applying a multi-objective evolutionary algorithm with an intensification operator to solve the problem. A simulated client–server architecture of four devices with four different processing speeds is studied. Both fully connected and convolutional neural network models are investigated with 33,400 and 887,530 weights, respectively. The experiments are performed using the MNIST and Fashion-MNIST datasets. A comparison is made between three approaches using an extensive set of metrics. Results prove that our approach obtains solutions with better accuracy than the full-communication setting and other methods while getting reductions in communications by around 1,000 times in most cases and up to 10,000 times in some cases compared to the maximum communication setting.

Adaptive client selection with personalization for communication efficient Federated Learning

Federated Select: A Primitive for Communication- and Memory-Efficient Federated Learning

Federated Learning with Additional Mechanisms on Clients to Reduce Communication Costs

DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training

A Multi-agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learning

FLAS: Computation and Communication Efficient Federated Learning via Adaptive Sampling

Communication Efficient Federated Learning With Heterogeneous Structured Client Models

Asynchronous Online Federated Learning with Reduced Communication Requirements

Active Client Selection for Clustered Federated Learning

Clients Eligibility-Based Lightweight Protocol in Federated Learning: An IDS Use Case

THF: 3-way Hierarchical Framework For Efficient Client Selection And Resource Management In Federated Learning

Latency Aware Semi-synchronous Client Selection and Model Aggregation for Wireless Federated Learning

SpaFL: Communication-Efficient Federated Learning with Sparse Models and Low computational Overhead

AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devices

Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning

A multi-objective approach for communication reduction in federated learning under devices heterogeneity constraints

Communication-Efficient Federated Learning in Channel Constrained Internet of Things

Enhancing Decentralized Federated Learning with Model Pruning and Adaptive Communication

Noise-Robust and Resource-Efficient ADMM-based Federated Learning

Towards Efficient Federated Learning Framework Via Selective Aggregation of Models

Towards Client Driven Federated Learning