Abstract:Federated Learning (FL) stands out as a widely adopted protocol facilitating the training of Machine Learning (ML) models while maintaining decentralized data. However, challenges arise when dealing with a heterogeneous set of participating devices, causing delays in the training process, particularly among devices with limited resources. Moreover, the task of training ML models with a vast number of parameters demands computing and memory resources beyond the capabilities of small devices, such as mobile and Internet of Things (IoT) devices. To address these issues, techniques like Parallel Split Learning (SL) have been introduced, allowing multiple resource-constrained devices to actively participate in collaborative training processes with assistance from resourceful compute nodes. Nonetheless, a drawback of Parallel SL is the substantial memory allocation required at the compute nodes, for instance training VGG-19 with 100 participants needs 80 GB. In this paper, we introduce Multihop Parallel SL (MP-SL), a modular and extensible ML as a Service (MLaaS) framework designed to facilitate the involvement of resource-constrained devices in collaborative and distributed ML model training. Notably, to alleviate memory demands per compute node, MP-SL supports multihop Parallel SL-based training. This involves splitting the model into multiple parts and utilizing multiple compute nodes in a pipelined manner. Extensive experimentation validates MP-SL's capability to handle system heterogeneity, demonstrating that the multihop configuration proves more efficient than horizontally scaled one-hop Parallel SL setups, especially in scenarios involving more cost-effective compute nodes.

Cluster-HSFL: A Cluster-Based Hybrid Split and Federated Learning.

Split Federated Learning Over Heterogeneous Edge Devices: Algorithm and Optimization

HSFL: an Efficient Split Federated Learning Framework Via Hierarchical Organization.

ParallelSFL: A Novel Split Federated Learning Framework Tackling Heterogeneity Issues

Hierarchical Federated Learning with Adaptive Clustering on Non-IID Data

Accelerating Split Federated Learning over Wireless Communication Networks

HASFL: Harnessing Heterogeneous Models Across Diverse Devices for Enhanced Federated Learning

Accelerating Hierarchical Federated Learning with Model Splitting in Edge Computing

HSFL: Efficient and Privacy-Preserving Offloading for Split and Federated Learning in IoT Services

Split Federated Learning: Speed up Model Training in Resource-Limited Wireless Networks

Clustered Federated Learning in Heterogeneous Environment.

Hierarchical Split Federated Learning: Convergence Analysis and System Optimization

FedCST: Federated Learning on Heterogeneous Resource-constrained Devices Using Clustering and Split Training

A Joint Gradient and Loss Based Clustered Federated Learning Design

MP-SL: Multihop Parallel Split Learning

SCFL: An Efficient Cross-cluster Federated Learning Framework Based on State Channels

Hybrid Architectures for Distributed Machine Learning in Heterogeneous Wireless Networks

Split-FL: An Efficient Online Federated Learning Framework with Constrained Computation and Streaming Data

Clustered Data Sharing for Non-IID Federated Learning over Wireless Networks

Have Your Cake and Eat It Too: Toward Efficient and Accurate Split Federated Learning

Accelerating Federated Learning with Cluster Construction and Hierarchical Aggregation.