Abstract:Federated learning (FL) is a privacy preserving machine learning paradigm designed to collaboratively learn a global model without data leakage. Specifically, in a typical FL system, the central server solely functions as an coordinator to iteratively aggregate the collected local models trained by each client, potentially introducing single-point transmission bottleneck and security threats. To mitigate this issue, decentralized federated learning (DFL) has been proposed, where all participating clients engage in peer-to-peer communication without a central server. Nonetheless, DFL still suffers from training degradation as FL does due to the non-independent and identically distributed (non-IID) nature of client data. And incorporating personalization layers into DFL may be the most effective solutions to alleviate the side effects caused by non-IID data. Therefore, in this paper, we propose a novel unit representation aided personalized decentralized federated learning framework, named UA-PDFL, to deal with the non-IID challenge in DFL. By adaptively adjusting the level of personalization layers through the guidance of the unit representation, UA-PDFL is able to address the varying degrees of data skew. Based on this scheme, client-wise dropout and layer-wise personalization are proposed to further enhance the learning performance of DFL. Extensive experiments empirically prove the effectiveness of our proposed method.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the decline in learning performance in Decentralized Federated Learning (DFL) due to the non - independent and identically distributed (non - IID) characteristics of data distribution. Specifically, the paper proposes a new framework named UA - PDFL (Unit Representation Aided Personalized Decentralized Federated Learning), aiming to address this challenge by introducing unit representation, client - wise dropout mechanism, and layer - wise personalization method. ### Summary of Main Problems 1. **Communication Cost and Security in Decentralized Federated Learning**: - In the DFL framework, all participating clients conduct peer - to - peer communication without a central server. Although this structure reduces the risk of single - point failure, it also increases communication overhead and may bring security risks. 2. **The non - IID Problem of Data Distribution**: - Clients' data are usually non - IID, which will lead to bias during model aggregation and then affect the overall learning performance. Although some existing Personalized Federated Learning (PFL) methods can alleviate this problem, most of them rely on public data or specific assumptions and are difficult to be effectively applied in the DFL environment. ### Solutions 1. **Unit Representation**: - The concept of unit representation is introduced to dynamically adjust the number of personalized layers, so as to effectively manage the degree of data skew among different clients. In this way, the subtle differences in clients' data distribution can be accurately captured without relying on public data, thereby reducing communication costs and enhancing system security. 2. **Client - wise Dropout**: - A client - wise dropout mechanism is developed. When the divergence between connected clients is lower than the specified threshold, one client is randomly retained. This method can effectively alleviate the over - fitting problem, especially when clients' data are relatively independent and identically distributed. 3. **Layer - wise Personalization**: - A layer - wise personalization method is proposed to guide the training of the general feature extractor and the personalized classifier, enabling clients to retain local classification bias while extracting features. This helps to improve the overall learning performance of DFL. ### Experimental Verification - The effectiveness of the proposed UA - PDFL method under various datasets and data distributions has been verified through extensive experiments. In conclusion, this paper solves the problem of learning performance decline in DFL due to the non - IID characteristics of data through innovative methods, providing a new solution for decentralized federated learning.

UA-PDFL: A Personalized Approach for Decentralized Federated Learning

Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training

Group Personalized Federated Learning

Personalizing or Not: Dynamically Personalized Federated Learning with Incentives

DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training

Decentralized Directed Collaboration for Personalized Federated Learning

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

Secure and Efficient Decentralized Federated Learning with Data Representation Protection

Personalized and privacy-enhanced federated learning framework via knowledge distillation

Optimizing Personalized Federated Learning through Adaptive Layer-Wise Learning

PLDP-FL: Federated Learning with Personalized Local Differential Privacy

Learn What You Need in Personalized Federated Learning

Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneity

PPFed: A Privacy-Preserving and Personalized Federated Learning Framework

FedL2P: Federated Learning to Personalize

Personalized Privacy-Preserving Framework for Cross-Silo Federated Learning

Personalized federated learning based on feature fusion

User-Centric Federated Learning: Trading off Wireless Resources for Personalization

Federated Deep Equilibrium Learning: Harnessing Compact Global Representations to Enhance Personalization

Advancing Personalized Federated Learning: Group Privacy, Fairness, and Beyond

Federated Learning with Personalization Layers