Abstract:Extreme resource constraints make large-scale machine learning (ML) with distributed clients challenging in wireless networks. On the one hand, large-scale ML requires massive information exchange between clients and server(s). On the other hand, these clients have limited battery and computation powers that are often dedicated to operational computations. Split federated learning (SFL) is emerging as a potential solution to mitigate these challenges, by splitting the ML model into client-side and server-side model blocks, where only the client-side block is trained on the client device. However, practical applications require personalized models that are suitable for the client's personal task. Motivated by this, we propose a personalized hierarchical split federated learning (PHSFL) algorithm that is specially designed to achieve better personalization performance. More specially, owing to the fact that regardless of the severity of the statistical data distributions across the clients, many of the features have similar attributes, we only train the body part of the federated learning (FL) model while keeping the (randomly initialized) classifier frozen during the training phase. We first perform extensive theoretical analysis to understand the impact of model splitting and hierarchical model aggregations on the global model. Once the global model is trained, we fine-tune each client classifier to obtain the personalized models. Our empirical findings suggest that while the globally trained model with the untrained classifier performs quite similarly to other existing solutions, the fine-tuned models show significantly improved personalized performance.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: in the resource - constrained wireless network environment, how to implement large - scale machine learning (ML) and ensure the personalized performance of the model. Specifically, the paper aims to address the following challenges: 1. **Resource Constraints**: Devices in wireless networks usually have limited computing and storage resources, which are often used to perform operational computing tasks and cannot be fully dedicated to training complex machine - learning models. 2. **Data Heterogeneity**: The data distribution of different clients varies greatly (non - independent and identically distributed, non - IID), which will lead to a decline in the performance of the global model, especially on statistically significantly different data sets. 3. **Communication Overhead**: Traditional federated learning (FL) requires a large amount of information exchange, which places higher demands on the communication between devices and servers in wireless networks. To solve these problems, the paper proposes a new algorithm - **Personalized Hierarchical Split Federated Learning (PHSFL)**. The main innovations of this algorithm include: - **Split Learning (SL)**: Divide the ML model into two parts, the client - side and the server - side, and only train a part of the model on the client - side to reduce the computing and communication burden. - **Keep the Classifier Frozen**: During the training process, keep the output layer (i.e., the classifier) unchanged and only train the feature extraction part of the model. In this way, a personalized model can be obtained by fine - tuning the classifier after global training. - **Hierarchical Aggregation**: By introducing a multi - layer structure of edge servers (ES) and central servers (CS), the aggregation process of model parameters is optimized, and the training efficiency and model performance are improved. Through the above methods, the PHSFL algorithm can achieve better personalized performance in a resource - constrained environment while reducing communication and computing overhead. Experimental results show that compared with the existing HSFL algorithm, PHSFL has a significant improvement in test accuracy and loss, especially in the case of severely skewed data distribution. In summary, by introducing the PHSFL algorithm, this paper effectively solves the large - scale machine - learning problem of resource - constrained devices in wireless networks and has made significant progress in personalized model performance.

Personalized Hierarchical Split Federated Learning in Wireless Networks

Hierarchical Federated Learning with Adaptive Clustering on Non-IID Data

HSFL: an Efficient Split Federated Learning Framework Via Hierarchical Organization.

Group Personalized Federated Learning

Sparse Federated Learning with Hierarchical Personalized Models

Hierarchical Personalized Federated Learning Over Massive Mobile Edge Computing Networks

PFSL: Personalized & Fair Split Learning with Data & Label Privacy for thin clients

HierSFL: Local Differential Privacy-aided Split Federated Learning in Mobile Edge Computing

Wireless Distributed Learning: A New Hybrid Split and Federated Learning Approach

Federated Split Learning Via Mutual Knowledge Distillation

A Comprehensive Study on Personalized Federated Learning with Non-IID Data.

Adaptive Personlization in Federated Learning for Highly Non-i.i.d. Data

Over-the-Air Hierarchical Personalized Federated Learning

Splitfed learning without client-side synchronization: Analyzing client-side split network portion size to overall performance

Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training

Hierarchical Split Federated Learning: Convergence Analysis and System Optimization

Adaptive Model Pruning and Personalization for Federated Learning Over Wireless Networks

User-Centric Federated Learning: Trading off Wireless Resources for Personalization

Accelerating Hierarchical Federated Learning with Model Splitting in Edge Computing

Hierarchical Personalized Federated Learning for User Modeling

Adaptive Hierarchical Federated Learning Over Wireless Networks