Abstract:Federated learning techniques provide a promising framework for collaboratively training a machine learning model without sharing users' data, and delivering a security solution to guarantee privacy during the model training of IoT devices. Nonetheless, challenges posed by data heterogeneity and communication resource constraints make it difficult to develop an efficient federated learning algorithm in terms of the low order of convergence rate. It could significantly deteriorate the quality of service for critical machine learning tasks, e.g., facial recognition, which requires an edge-ready, low-power, low-latency training algorithm. To address these challenges, a communication-efficient federated learning approach is proposed in this paper where the momentum technique is leveraged to accelerate the convergence rate while largely reducing the communication requirements. First, a federated multi-task learning framework by which the learning tasks are reformulated by the multi-objective optimization problem is introduced to address the data heterogeneity. The multiple gradient descent algorithm is harnessed to find the common gradient descending direction for all participants so that the common features can be learned and no sacrifice on each clients' performance. Second, to reduce communication costs, a local momentum technique with global information is developed to speed up the convergence rate, where the convergence analysis of the proposed method under non-convex case is studied. It is proved that the proposed local momentum can actually achieve the same acceleration as the global momentum, whereas it is more robust than algorithms that solely rely on the acceleration by the global momentum. Third, the generalization of the proposed acceleration approach is investigated which is demonstrated by the accelerated variation of FedAvg. Finally, the performance of the proposed method on the learning model accuracy, convergence rate, and robustness to data heterogeneity, is investigated by empirical experiments on four public datasets, while a real-world IoT platform is constructed to demonstrate the communication efficiency of the proposed method.

Decentralized federated meta‐learning framework for few‐shot multitask learning

HFedMTL: Hierarchical Federated Multi-Task Learning

DFML: Decentralized Federated Mutual Learning

FewFedWeight: Few-shot Federated Learning Framework Across Multiple NLP Tasks

TinyMetaFed: Efficient Federated Meta-Learning for TinyML

Federated Few-Shot Learning with Adversarial Learning

Heterogeneous Federated Meta-Learning with Mutually Constrained Propagation

A Decentralized Federated Learning Framework Via Committee Mechanism with Convergence Guarantee

Optimizing Multimodal Federated Learning: Novel Approaches for Efficient Model Aggregation and Client Sampling

Secure Collaborative Few-Shot Learning

Federated Multitask Learning for HyperFace

3FM: Multi-modal Meta-learning for Federated Tasks

Few-Shot Model Agnostic Federated Learning

Meta-Learning across Meta-Tasks for Few-Shot Learning

Few-Shot Federated Learning: A Federated Learning Model for Small-Sample Scenarios

Federated Meta-Learning: Methodologies and Directions

DeceFL:a Principled Fully Decentralized Federated Learning Framework

Accelerating Communication-efficient Federated Multi-Task Learning with Personalization and Fairness

DeFTA: A Plug-and-Play Peer-to-Peer Decentralized Federated Learning Framework

Federated Meta-Learning for Few-Shot Fault Diagnosis With Representation Encoding

Online Federated Multitask Learning.