Abstract:Federated Learning (FL) enables multiple devices to collaboratively train a shared model while preserving data privacy. Ever-increasing model complexity coupled with limited memory resources on the participating devices severely bottlenecks the deployment of FL in real-world scenarios. Thus, a framework that can effectively break the memory wall while jointly taking into account the hardware and statistical heterogeneity in FL is urgently required. In this paper, we propose SmartSplit, a framework that effectively reduces the memory footprint on the device side while guaranteeing the training progress and model accuracy for heterogeneous FL through model <a class="link-external link-http" href="http://splitting.Towards" rel="external noopener nofollow">this http URL</a> this end, SmartSplit employs a hierarchical structure to adaptively guide the overall training process. In each training round, the central manager, hosted on the server, dynamically selects the participating devices and sets the cutting layer by jointly considering the memory budget, training capacity, and data distribution of each device. The MEC manager, deployed within the edge server, proceeds to split the local model and perform training of the server-side portion. Meanwhile, it fine-tunes the splitting points based on the time-evolving statistical importance. The on-device manager, embedded inside each mobile device, continuously monitors the local training status while employing cost-aware checkpointing to match the runtime dynamic memory budget. Extensive experiments on representative datasets are conducted on both commercial off-the-shelf mobile device testbeds. The experimental results show that SmartSplit excels in FL training on highly memory-constrained mobile SoCs, offering up to a 94% peak latency reduction and 100-fold memory savings. It enhances accuracy performance by 1.49%-57.18% and adaptively adjusts to dynamic memory budgets through cost-aware recomputation.

Efficient Split Learning with Non-iid Data

Towards Efficient Scheduling of Federated Mobile Devices under Computational and Statistical Heterogeneity

An Efficient Split Learning Framework for Recurrent Neural Network in Mobile Edge Environment.

Resource-efficient Parallel Split Learning in Heterogeneous Edge Computing

Split Federated Learning Over Heterogeneous Edge Devices: Algorithm and Optimization

Efficient Collaborative Learning over Unreliable D2D Network: Adaptive Cluster Head Selection and Resource Allocation

Split Learning Over Wireless Networks: Parallel Design and Resource Management

Accelerating Split Federated Learning over Wireless Communication Networks

Dynamic Encoding and Decoding of Information for Split Learning in Mobile-Edge Computing: Leveraging Information Bottleneck Theory

Split-FL: An Efficient Online Federated Learning Framework with Constrained Computation and Streaming Data

Splitfed learning without client-side synchronization: Analyzing client-side split network portion size to overall performance

Parallel Split Learning with Global Sampling

Learning the Optimal Partition for Collaborative DNN Training with Privacy Requirements

Communication-efficient federated continual learning for distributed learning system with Non-IID data

A Novel Neural Network-Based Federated Learning System for Imbalanced and Non-IID Data

Split Learning for Distributed Collaborative Training of Deep Learning Models in Health Informatics

HierSFL: Local Differential Privacy-aided Split Federated Learning in Mobile Edge Computing

Communication and Computation Reduction for Split Learning using Asynchronous Training

Breaking the Memory Wall for Heterogeneous Federated Learning via Model Splitting

Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networks

Efficient Communication-Computation Tradeoff for Split Computing: A Multi-Tier Deep Reinforcement Learning Approach