Abstract:Federated learning techniques provide a promising framework for collaboratively training a machine learning model without sharing users' data, and delivering a security solution to guarantee privacy during the model training of IoT devices. Nonetheless, challenges posed by data heterogeneity and communication resource constraints make it difficult to develop an efficient federated learning algorithm in terms of the low order of convergence rate. It could significantly deteriorate the quality of service for critical machine learning tasks, e.g., facial recognition, which requires an edge-ready, low-power, low-latency training algorithm. To address these challenges, a communication-efficient federated learning approach is proposed in this paper where the momentum technique is leveraged to accelerate the convergence rate while largely reducing the communication requirements. First, a federated multi-task learning framework by which the learning tasks are reformulated by the multi-objective optimization problem is introduced to address the data heterogeneity. The multiple gradient descent algorithm is harnessed to find the common gradient descending direction for all participants so that the common features can be learned and no sacrifice on each clients' performance. Second, to reduce communication costs, a local momentum technique with global information is developed to speed up the convergence rate, where the convergence analysis of the proposed method under non-convex case is studied. It is proved that the proposed local momentum can actually achieve the same acceleration as the global momentum, whereas it is more robust than algorithms that solely rely on the acceleration by the global momentum. Third, the generalization of the proposed acceleration approach is investigated which is demonstrated by the accelerated variation of FedAvg. Finally, the performance of the proposed method on the learning model accuracy, convergence rate, and robustness to data heterogeneity, is investigated by empirical experiments on four public datasets, while a real-world IoT platform is constructed to demonstrate the communication efficiency of the proposed method.

HFedMTL: Hierarchical Federated Multi-Task Learning

RHFedMTL: Resource-Aware Hierarchical Federated Multi-Task Learning

Hierarchical federated learning based on wireless D2D networks

Federated Multitask Learning for HyperFace

Enhanced Hybrid Hierarchical Federated Edge Learning Over Heterogeneous Networks

Hierarchical Federated Learning with Adaptive Clustering on Non-IID Data

FedHCA$^2$: Towards Hetero-Client Federated Multi-Task Learning

Online Federated Multitask Learning.

FedBone: Towards Large-Scale Federated Multi-Task Learning

HFML: Heterogeneous Hierarchical Federated Mutual Learning on Non-Iid Data

Federated Multi-Task Learning with Non-Stationary and Heterogeneous Data in Wireless Networks

Federated Multi-task Graph Learning

Communication-Efficient Federated Multi-Task Learning with Sparse Sharing

FedDGA: Federated Multi-Task Learning Based on Dynamic Guided Attention

Federated Multiple Label Hashing (FedMLH): Communication Efficient Federated Learning on Extreme Classification Tasks

HSFL: an Efficient Split Federated Learning Framework Via Hierarchical Organization.

Efficient Federated Multi-view Learning

Federated Multi-Task Learning under a Mixture of Distributions

Adaptive Training and Aggregation for Federated Learning in Multi-tier Computing Networks

Decentralized federated meta‐learning framework for few‐shot multitask learning

Accelerating Communication-efficient Federated Multi-Task Learning with Personalization and Fairness