Abstract:Efficient collaboration between collaborative machine learning and wireless communication technology, forming a Federated Edge Learning (FEEL), has spawned a series of next-generation intelligent applications. However, due to the openness of network connections, the FEEL framework generally involves hundreds of remote devices (or clients), resulting in expensive communication costs, which is not friendly to resource-constrained FEEL. To address this issue, we propose a distributed approximate Newton-type algorithm with fast convergence speed to alleviate the problem of FEEL resource (in terms of communication resources) constraints. Specifically, the proposed algorithm is improved based on distributed L-BFGS algorithm and allows each client to approximate the high-cost Hessian matrix by computing the low-cost Fisher matrix in a distributed manner to find a better descent direction, thereby speeding up convergence. Second, we prove that the proposed algorithm has linear convergence in strongly convex and non-convex cases and analyze its computational and communication complexity. Similarly, due to the heterogeneity of the connected remote devices, FEEL faces the challenge of heterogeneous data and non-IID (Independent and Identically Distributed) data. To this end, we design a simple but elegant training scheme, namely FedOVA (Federated One-vs-All), to solve the heterogeneous statistical challenge brought by heterogeneous data. In this way, FedOVA first decomposes a multi-class classification problem into more straightforward binary classification problems and then combines their respective outputs using ensemble learning. In particular, the scheme can be well integrated with our communication efficient algorithm to serve FEEL. Numerical results verify the effectiveness and superiority of the proposed algorithm.

Resource Aware Clustering for Tackling the Heterogeneity of Participants in Federated Learning

Resource Aware Clustering for Tackling the Heterogeneity of Participants in Federated Learning

TKAGFL: A Federated Communication Framework under Data Heterogeneity

Hierarchical Federated Learning with Adaptive Clustering on Non-IID Data

An Efficient Framework for Clustered Federated Learning

Fair Selection of Edge Nodes to Participate in Clustered Federated Multitask Learning

Fair Federated Data Clustering through Personalization: Bridging the Gap between Diverse Data Distributions

Resource-constrained Federated Learning with Heterogeneous Data: Formulation and Analysis

Faster Convergence on Heterogeneous Federated Edge Learning: An Adaptive Clustered Data Sharing Approach

Adaptive Clustered Federated Learning with Representation Similarity

Equitable Federated Learning with Activation Clustering

Federated learning with incremental clustering for heterogeneous data

Mitigating Disparate Impact of Differential Privacy in Federated Learning through Robust Clustering

Dynamic User Clustering for Efficient and Privacy-Preserving Federated Learning

Heterogeneity for the Win: One-Shot Federated Clustering

Heterogeneity-aware device selection for clustered federated learning in IoT

FedClust: Tackling Data Heterogeneity in Federated Learning through Weight-Driven Client Clustering

Dual-Segment Clustering Strategy for Hierarchical Federated Learning in Heterogeneous Wireless Environments

Personalized Federated Learning with Multiple Known Clusters

A Cluster-Driven Adaptive Training Approach for Federated Learning

Privacy-preserving clustering federated learning for non-IID data