Abstract:Recent advancements in decentralized learning, such as Federated Learning (FL), Split Learning (SL), and Split Federated Learning (SplitFed), have expanded the potentials of machine learning. SplitFed aims to minimize the computational burden on individual clients in FL and parallelize SL while maintaining privacy. This study investigates the resilience of SplitFed to packet loss at model split points. It explores various parameter aggregation strategies of SplitFed by examining the impact of splitting the model at different points-either shallow split or deep split-on the final global model performance. The experiments, conducted on a human embryo image segmentation task, reveal a statistically significant advantage of a deeper split point.

What problem does this paper attempt to address?

This paper aims to solve the problem of the robustness of SplitFed learning (Split Federated Learning) against packet loss at the model split point. Specifically, the main research objectives include: 1. **Explore the impact of different model split point selections on SplitFed learning**: The paper experimentally analyzes the impact of shallow split and deep split on the performance of the final global model. 2. **Evaluate the performance of different parameter aggregation strategies in the presence of packet loss**: The paper examines the effects of multiple parameter aggregation methods (such as naive averaging, federated averaging, etc.) under different packet loss probabilities (\(PL\)) and the number of clients experiencing packet loss (\(N_c\)). ### Research Background - **Federated Learning (FL)**: Allows multiple clients to collaboratively train machine - learning models without sharing data, especially suitable for privacy - sensitive fields such as healthcare. - **Split Learning (SL)**: Reduces the computational burden on clients by splitting the model between the client and the server. - **Split Federated Learning (SplitFed)**: Combines the privacy protection of FL and the model - balancing advantages of SL, aiming to minimize the computational burden on clients and maintain privacy. ### Research Problem Although previous studies have explored the robustness of SplitFed against labeling errors and communication noise, no in - depth study has been specifically carried out on packet loss. Packet loss is a common transmission error in distributed learning, especially at the model split point, which will affect the selection of the optimal split point. ### Experimental Design - **Model Selection**: Use the Split U - Net model for human embryo image segmentation tasks. - **Dataset**: Adopt the Blastocyst dataset, which contains 781 human embryo images and their segmentation masks. - **Experimental Setup**: - Packet loss probability \(PL\in\{0.1, 0.3, 0.5, 0.7, 0.9\}\) - The number of clients experiencing packet loss \(N_c\in\{0, 1, 2, 3, 4, 5\}\) - Parameter aggregation methods include: naive averaging, federated averaging (FedAvg), auto - federated averaging (auto - FedAvg), fed - NCL V2 and fed - NCL V4. ### Main Findings - **Performance without packet loss**: Experiments show that, regardless of the parameter aggregation method used, the performance of the SplitFed U - Net model is better than that of the centrally - trained U - Net model. - **Performance in the presence of packet loss**: The deep - split model is significantly better than the shallow - split model, especially under high packet loss probabilities. Deep - split recovers lost data by adding additional network layers, and the initial skip connection is completely located on the client side, avoiding the impact of packet loss. ### Conclusion This study shows that in the presence of packet loss, the selection of the deep - split point can significantly improve the robustness and performance of SplitFed learning. Future research will further explore multiple SplitFed networks, more realistic packet loss models, and develop more robust aggregation and recovery methods.

Optimizing Split Points for Error-Resilient SplitFed Learning

SplitFed resilience to packet loss: Where to split, that is the question

Splitfed learning without client-side synchronization: Analyzing client-side split network portion size to overall performance

Quality-Adaptive Split-Federated Learning for Segmenting Medical Images with Inaccurate Annotations

Accelerating Split Federated Learning over Wireless Communication Networks

Split Federated Learning Over Heterogeneous Edge Devices: Algorithm and Optimization

FedEmb: A Vertical and Hybrid Federated Learning Algorithm using Network And Feature Embedding Aggregation

AdaSplit: Adaptive Trade-offs for Resource-constrained Distributed Deep Learning

Split-FL: An Efficient Online Federated Learning Framework with Constrained Computation and Streaming Data

SplitAVG: A Heterogeneity-Aware Federated Deep Learning Method for Medical Imaging

Heterogeneous Federated Learning with Splited Language Model

When Computing Power Network Meets Distributed Machine Learning: An Efficient Federated Split Learning Framework

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

Hierarchical Split Federated Learning: Convergence Analysis and System Optimization

Have Your Cake and Eat It Too: Toward Efficient and Accurate Split Federated Learning

Communication-Efficient Split Learning via Adaptive Feature-Wise Compression

Accelerating Hierarchical Federated Learning with Model Splitting in Edge Computing

Federated Split Learning for Edge Intelligence in Resource-Constrained Wireless Networks

Training Latency Minimization for Model-Splitting Allowed Federated Edge Learning

AdaptSFL: Adaptive Split Federated Learning in Resource-constrained Edge Networks

When MiniBatch SGD Meets SplitFed Learning:Convergence Analysis and Performance Evaluation