Abstract:Federated learning is a paradigm that proposes protecting data privacy by sharing local models instead of raw data during each iteration of model training. However, these models can be large, with many parameters, provoking a substantial communication cost and having a notable environmental impact. Reducing communication overhead is paramount but conflictual to maintaining the model's accuracy. Most research has dealt with the different factors influencing communication reduction separately without addressing their correlations. Moreover, most of them do not consider the heterogeneity of clients' hardware. Finding the optimal configuration to fulfil all these training aspects can become intractable for classical techniques. This work explores the add-in that multi-objective evolutionary algorithms can provide for solving the communication overhead problem while achieving high accuracy. We do this by 1) realistically modelling and formulating this task as a multi-objective problem by considering the devices' heterogeneity, 2) including all the communication-triggering aspects, and 3) applying a multi-objective evolutionary algorithm with an intensification operator to solve the problem. A simulated client–server architecture of four devices with four different processing speeds is studied. Both fully connected and convolutional neural network models are investigated with 33,400 and 887,530 weights, respectively. The experiments are performed using the MNIST and Fashion-MNIST datasets. A comparison is made between three approaches using an extensive set of metrics. Results prove that our approach obtains solutions with better accuracy than the full-communication setting and other methods while getting reductions in communications by around 1,000 times in most cases and up to 10,000 times in some cases compared to the maximum communication setting.

Two-Stream Federated Learning: Reduce the Communication Costs

Federated Learning with Additional Mechanisms on Clients to Reduce Communication Costs

Message Passing Based Wireless Federated Learning Via Analog Message Aggregation

Hierarchical federated learning based on wireless D2D networks

Communication-efficient federated continual learning for distributed learning system with Non-IID data

Federated Learning: Strategies for Improving Communication Efficiency

Hierarchical Federated Learning with Adaptive Clustering on Non-IID Data

Federated Learning in Multi-RIS-Aided Systems

Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning

Limitations and Future Aspects of Communication Costs in Federated Learning: A Survey

Enhancing Decentralized Federated Learning with Model Pruning and Adaptive Communication

Communication-Efficient Model Aggregation with Layer Divergence Feedback in Federated Learning

Communication Efficient Federated Learning With Heterogeneous Structured Client Models

A multi-objective approach for communication reduction in federated learning under devices heterogeneity constraints

Differential Privacy Meets Federated Learning under Communication Constraints

Computation and Communication Efficient Federated Learning With Adaptive Model Pruning

Towards Faster And Better Federated Learning: A Feature Fusion Approach

A Communication-Efficient Hierarchical Federated Learning Framework via Shaping Data Distribution at Edge

Federated Learning via Over-the-Air Computation

Edge-Based Communication Optimization for Distributed Federated Learning

FLAS: Computation and Communication Efficient Federated Learning via Adaptive Sampling