Abstract:Semi-decentralized federated learning blends the conventional device-to-server (D2S) interaction structure of federated model training with localized device-to-device (D2D) communications. We study this architecture over edge networks with multiple D2D clusters modeled as time-varying and directed communication graphs. Our investigation results in two algorithms: (a) a connectivity-aware learning algorithm that controls the fundamental trade-off between the convergence rate of the model training process and the number of energy-intensive D2S transmissions required for global aggregation, and (b) a motion-planning algorithm to enhance the densities and regularity levels of cluster digraphs so as to further reduce the number of D2S transmissions in connectivity-aware learning. Specifically, in our semi-decentralized methodology, weighted-averaging-based D2D updates are injected into the federated averaging framework based on column-stochastic weight matrices that encapsulate the connectivity within the clusters. To develop our algorithm, we show how the current expected optimality gap (i.e., the distance between the most recent global model computed by the server and the target/desired optimal model) depends on the greatest two singular values of the weighted adjacency matrices (and hence on the densities and degrees of digraph regularity) of the D2D clusters. We then derive tight bounds on these singular values in terms of the node degrees of the D2D clusters, and we use the resulting expressions to design our connectivity-aware learning algorithm. Simulations performed using real-world datasets and Random Direction Mobility Model (RDMM)-based time-varying D2D topologies reveal that our connectivity-aware algorithm significantly reduces the total communication energy required to reach a target accuracy level compared with baselines while achieving the accuracy level in nearly the same number of iterations as these baselines.

Distributed Learning for Time-varying Networks: A Scalable Design

Distributed Learning in Wireless Networks: Recent Progress and Future Challenges

Distributed Learning for Wireless Communications: Methods, Applications and Challenges

Distributed Learning Systems with First-order Methods

Asynchronous Federated Learning over Wireless Communication Networks.

Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications

Hierarchical federated learning based on wireless D2D networks

Scalable Learning Paradigms for Data-Driven Wireless Communication

Distributed estimation over complex networks

Distributed Learning of Predictive Structures from Multiple Tasks over Networks

Distributed Extreme Learning Machine for Nonlinear Learning over Network

Energy-Efficient Connectivity-Aware Learning Over Time-Varying D2D Networks

Accelerated Distributed Stochastic Non-Convex Optimization over Time-Varying Directed Networks

Distributed Machine Learning with Strategic Network Design: A Game-Theoretic Perspective

Over-the-air Learning Rate Optimization for Federated Learning

Distributed ADMM for Time-Varying Communication Networks

A Finite Time Discrete Distributed Learning Algorithm Using Stochastic Configuration Network

Towards Scalable Wireless Federated Learning: Challenges and Solutions

Decentralized learning for wireless communications and networking

Distributed Experimental Design Networks

Distributed Learning in Wireless Sensor Networks