Abstract:Federated Learning allows multiple participants to jointly train a global model while guaranteeing the confidentiality and integrity of private datasets. However, current server aggregation algorithms for federated learning only focus on model parameters, resulting in heavy communication costs and low convergence speed. Most importantly, they are unable to handle the scenario wherein different clients hold different local models with various network architectures. In this paper, we view these challenges from an alternative perspective: we draw attention to what should be aggregated and how to improve convergence efficiency. Specifically, we propose MHAT, a novel model-heterogenous aggregation training federated learning scheme which exploits a technique of Knowledge Distillation (KD) to extract the update information of the heterogenous model of all clients and trains an auxiliary model on the server to realize information aggregation. MHAT relaxes clients from fixing on an unified model architecture and significantly reduces the required computing resources while maintaining acceptable model convergence accuracy. Various experiments verify the effectiveness and applicability of our proposed scheme. (c) 2021 Elsevier Inc. All rights reserved. Federated learning 11?3] is an emerging machine learning paradigm for decentralized data 14,5], which enables multiple parties to collaboratively train a global model without sharing their private data. In the canonical federated learning protocol 16], model parameter is the interactive information between clients and the server. At the beginning of each training round, the central server distributes current global model to all participants, and all participants update the local model using their private data. Then the server will collect updated model information from all parties and obtain the central global model via a weighted average aggregation algorithm. However, in a practical federated learning, several major challenges still remain unsolved when using model parameters

MHAT: an Efficient Model-Heterogenous Aggregation Training Scheme for Federated Learning.

FedDGP: Disentangling Global and Personal Models for Federated Learning

Robust Model Aggregation for Federated Learning with Heterogeneous Clients

AsyncFedED: Asynchronous Federated Learning with Euclidean Distance Based Adaptive Weight Aggregation

A hierarchical federated learning model with adaptive model parameter aggregation

FedHCA$^2$: Towards Hetero-Client Federated Multi-Task Learning

An Aggregation-Free Federated Learning for Tackling Data Heterogeneity

Is Aggregation the Only Choice? Federated Learning via Layer-wise Model Recombination

Handling Data Heterogeneity in Federated Learning via Knowledge Distillation and Fusion

HFML: Heterogeneous Hierarchical Federated Mutual Learning on Non-Iid Data

An Efficient Multi-Model Training Algorithm for Federated Learning

Takle the Data Heterogeneous Variations in Federated Learning Via Adaptive Aggregate Weights

Federated Learning Model Training Method Based on Data Features Perception Aggregation

Lazy Aggregation for Heterogeneous Federated Learning

A Hierarchical Knowledge Transfer Framework for Heterogeneous Federated Learning.

Dual-Criterion Model Aggregation in Federated Learning: Balancing Data Quantity and Quality

CSMAAFL: Client Scheduling and Model Aggregation in Asynchronous Federated Learning

FedCav: Contribution-aware Model Aggregation on Distributed Heterogeneous Data in Federated Learning

An Efficient and Robust Aggregation Algorithm for Learning Federated CNN.

Robust Heterogeneous Federated Learning Via Data-Free Knowledge Amalgamation.

Distributed Learning in Heterogeneous Environment: federated learning with adaptive aggregation and computation reduction