Abstract:Federated learning techniques provide a promising framework for collaboratively training a machine learning model without sharing users' data, and delivering a security solution to guarantee privacy during the model training of IoT devices. Nonetheless, challenges posed by data heterogeneity and communication resource constraints make it difficult to develop an efficient federated learning algorithm in terms of the low order of convergence rate. It could significantly deteriorate the quality of service for critical machine learning tasks, e.g., facial recognition, which requires an edge-ready, low-power, low-latency training algorithm. To address these challenges, a communication-efficient federated learning approach is proposed in this paper where the momentum technique is leveraged to accelerate the convergence rate while largely reducing the communication requirements. First, a federated multi-task learning framework by which the learning tasks are reformulated by the multi-objective optimization problem is introduced to address the data heterogeneity. The multiple gradient descent algorithm is harnessed to find the common gradient descending direction for all participants so that the common features can be learned and no sacrifice on each clients' performance. Second, to reduce communication costs, a local momentum technique with global information is developed to speed up the convergence rate, where the convergence analysis of the proposed method under non-convex case is studied. It is proved that the proposed local momentum can actually achieve the same acceleration as the global momentum, whereas it is more robust than algorithms that solely rely on the acceleration by the global momentum. Third, the generalization of the proposed acceleration approach is investigated which is demonstrated by the accelerated variation of FedAvg. Finally, the performance of the proposed method on the learning model accuracy, convergence rate, and robustness to data heterogeneity, is investigated by empirical experiments on four public datasets, while a real-world IoT platform is constructed to demonstrate the communication efficiency of the proposed method.

Research on Multi-source Data Security Fusion Analysis Technology Based on Federated Learning

Research on data fusion method based on Federated learning

Analysis of Heterogeneous Data Model Based on Federated Learning

A Multi-Source Credit Data Fusion Approach Based on Federated Distillation Learning

Federated Learning Algorithm Based on Adaptive Gradient Fusion

Federated Learning with Data-Agnostic Distribution Fusion

Multimodal Fusion with Block Term Decomposition for Asynchronous Federated Learning

FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation

A Survey on Federated Learning Technology.

Towards Faster And Better Federated Learning: A Feature Fusion Approach

A Secure and Fair Federated Learning Framework Based on Consensus Incentive Mechanism

Distinguishing Good from Bad: Distributed-Collaborative-Representation-Based Data Fraud Detection in Federated Learning.

From distributed machine learning to federated learning: In the view of data privacy and security

FEDDRL: TRUSTWORTHY FEDERATED LEARNING MODEL FUSION METHOD BASED ON STAGED REINFORCEMENT LEARNING

Federated Learning Meets Multi-objective Optimization

Accelerating Communication-efficient Federated Multi-Task Learning with Personalization and Fairness

FedSKG: Self-supervised Federated Learning with Structural Knowledge of Global Model

Federated Learning via Consensus Mechanism on Heterogeneous Data: A New Perspective on Convergence

Towards Efficient Federated Learning Framework Via Selective Aggregation of Models

A dynamic receptive field and improved feature fusion approach for federated learning in financial credit risk assessment

A Privacy-Enhanced Federated Learning Model Training Method