Abstract:Fully decentralized learning is gaining momentum for training AI models at the Internet's edge, addressing infrastructure challenges and privacy concerns. In a decentralized machine learning system, data is distributed across multiple nodes, with each node training a local model based on its respective dataset. The local models are then shared and combined to form a global model capable of making accurate predictions on new data. Our exploration focuses on how different types of network structures influence the spreading of knowledge - the process by which nodes incorporate insights gained from learning patterns in data available on other nodes across the network. Specifically, this study investigates the intricate interplay between network structure and learning performance using three network topologies and six data distribution methods. These methods consider different vertex properties, including degree centrality, betweenness centrality, and clustering coefficient, along with whether nodes exhibit high or low values of these metrics. Our findings underscore the significance of global centrality metrics (degree, betweenness) in correlating with learning performance, while local clustering proves less predictive. We highlight the challenges in transferring knowledge from peripheral to central nodes, attributed to a dilution effect during model aggregation. Additionally, we observe that central nodes exert a pull effect, facilitating the spread of knowledge. In examining degree distribution, hubs in Barabasi-Albert networks positively impact learning for central nodes but exacerbate dilution when knowledge originates from peripheral nodes. Finally, we demonstrate the formidable challenge of knowledge circulation outside of segregated communities.

Topology Construction with Minimum Total Time for Geo-Distributed Decentralized Federated Learning.

Enhancing Decentralized and Personalized Federated Learning with Topology Construction

Topology Learning for Heterogeneous Decentralized Federated Learning over Unreliable D2D Networks

Towards Practical Overlay Networks for Decentralized Federated Learning

Bandwidth Constrained Client Selection and Scheduling for Federated Learning over SD-WAN

Optimizing Communication Topology for Collaborative Learning Across Datacenters

Hierarchical federated learning based on wireless D2D networks

Impact of network topology on the performance of Decentralized Federated Learning

Beyond the Federation: Topology-aware Federated Learning for Generalization to Unseen Clients

Decentralized Federated Learning under Communication Delays.

EFFICIENT AND RELIABLE OVERLAY NETWORKS FOR DECENTRALIZED FEDERATED LEARNING\ast

D-Cliques: Compensating for Data Heterogeneity with Topology in Decentralized Federated Learning

Joint Participant and Learning Topology Selection for Federated Learning in Edge Clouds

Impact of Network Topology on Byzantine Resilience in Decentralized Federated Learning

FedTAD: Topology-aware Data-free Knowledge Distillation for Subgraph Federated Learning

Joint Topology and Computation Resource Optimization for Federated Edge Learning.

Time-Efficient Ring-Topology Design for Decentralized Learning Via Device-to-Device Communications

DeFed: A Principled Decentralized and Privacy-Preserving Federated Learning Algorithm.

Accelerating Decentralized Federated Learning in Heterogeneous Edge Computing

Accelerating Federated Edge Learning Via Topology Optimization