Abstract:With the recent progress in graph-federated learning (GFL), it has demonstrated a promising performance in effectively addressing challenges associated with heterogeneous clients. Although the majority of advances in GFL have been focused on techniques for elucidating the intricate relationships among clients, existing GFL methods have two limitations. First, current methods comprising the use of low-dimensional graphs fail to accurately depict the associations between clients, thereby compromising the performance of GFL. Second, these methods may disclose additional information when sharing client-side hidden representations. This paper presents a structural GFL (SGFL) framework and a suite of novel optimization methods. SGFL addresses the limitations of existing GFL approaches with three original contributions. Firstly, our approach advocates the dynamic construction of federated learning (FL) graphs by leveraging the high-dimensional information inherent among clients, while enabling the discovery of hierarchical communities within clients. Secondly, we present SG-FedX, a novel federated stochastic gradient optimization algorithm that mitigates the effects of heterogeneity by intelligently using a global representation. Furthermore, SG-FedX introduces a strict sharing mechanism that protects client privacy more effectively by refraining from sharing client information beyond the model parameters. Our comparative evaluations, conducted against ten representative FL algorithms under challenging non-independently-and-identically-distributed settings, demonstrated the superior performance of SG-FedX. It was noted that, in the cross-dataset scenarios, SG-FedX outperformed the second-best baseline by 8.12% and 7.91% in personalization and generalization performance, respectively.

Federated Sparse Gaussian Processes

FedDGP: Disentangling Global and Personal Models for Federated Learning

Federated Automatic Latent Variable Selection in Multi-output Gaussian Processes

Gradient-Congruity Guided Federated Sparse Training

Heterogeneous Federated Learning via Grouped Sequential-to-Parallel Training

Federated Graph Learning with Graphless Clients

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

Federated Learning for Non-factorizable Models using Deep Generative Prior Approximations

Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach

Scalable, Federated Gaussian Process Training for Decentralized Multi-Agent Systems

FedSSP: Federated Graph Learning with Spectral Knowledge and Personalized Preference

Personalized Federated Learning with Multi-View Geometry Structure

Efficient Personalized Federated Learning via Sparse Model-Adaptation

Achieving Personalized Federated Learning with Sparse Local Models

Federated Graph Learning -- A Position Paper

GPFL: A Gradient Projection-Based Client Selection Framework for Efficient Federated Learning

Structural graph federated learning: Exploiting high-dimensional information of statistical heterogeneity

FedGP: Correlation-Based Active Client Selection for Heterogeneous Federated Learning

Federated Learning with Neural Graphical Models

FedGSNR: Accelerating Federated Learning on Non-IID Data via Maximum Gradient Signal to Noise Ratio

Federated Stochastic Gradient Langevin Dynamics