Abstract:Personalized Federated Learning (PFL) focuses on tailoring models to individual IIoT clients in federated learning by addressing data heterogeneity and diverse user needs. Although existing studies have proposed effective PFL solutions from various perspectives, they overlook the issue of forgetting both historical personalized knowledge and global generalized knowledge during local training on clients. Therefore, this study proposes a novel PFL method, Federated Progressive Self-Distillation (FedPSD), based on logits calibration and progressive self-distillation. We analyze the impact mechanism of client data distribution characteristics on personalized and global knowledge forgetting. To address the issue of global knowledge forgetting, we propose a logits calibration approach for the local training loss and design a progressive self-distillation strategy to facilitate the gradual inheritance of global knowledge, where the model outputs from the previous epoch serve as virtual teachers to guide the training of subsequent epochs. Moreover, to address personalized knowledge forgetting, we construct calibrated fusion labels by integrating historical personalized model outputs, which are then used as teacher model outputs to guide the initial epoch of local self-distillation, enabling rapid recall of personalized knowledge. Extensive experiments under various data heterogeneity scenarios demonstrate the effectiveness and superiority of the proposed FedPSD method.

What problem does this paper attempt to address?

This paper attempts to solve two key problems in Personalized Federated Learning (PFL): **global knowledge forgetting** and **historical personalized knowledge forgetting**. Specifically: 1. **Global knowledge forgetting**: In traditional federated learning, each client may gradually forget the general knowledge learned from the global model during the local training process. This is because the data distribution of clients is usually non - independent and identically distributed (Non - IID), causing the local model to be biased towards adapting to local data characteristics during the optimization process, thus deviating from the generalization ability of the global model. 2. **Historical personalized knowledge forgetting**: As the number of training rounds increases, the client may forget the historical personalized knowledge learned before. Especially in the actual IIoT (Industrial Internet of Things) environment, due to unstable network connections, limited device resources, or strict local privacy policies, some clients may not be able to participate in each round of training regularly, which further exacerbates the forgetting of personalized knowledge. To solve these problems, the author proposes a new PFL method - **Federated Progressive Self - Distillation (FedPSD)**, which combines **logits calibration** and **progressive self - distillation** techniques. Specifically: - **Global knowledge forgetting**: By introducing logits calibration to dynamically adjust the output logits during the local training process, the loss of global knowledge is alleviated. At the same time, a progressive self - distillation strategy is adopted, so that the model output of the previous epoch serves as a virtual teacher to guide the training of subsequent epochs, thereby gradually inheriting global knowledge. - **Historical personalized knowledge forgetting**: By constructing calibrated fusion labels, the output of the historical personalized model is combined with the real label to guide the self - distillation process in the initial epoch, thereby quickly recovering personalized knowledge. ### Main contributions 1. **Theoretical analysis**: It is proved that the alternating process of aggregation and local training in federated learning will lead to continuous global and personalized knowledge forgetting. 2. **Innovative mechanism**: A progressive self - distillation mechanism and a dynamic logits calibration technique are proposed to meet the challenge of global knowledge forgetting caused by Non - IID data in the IIoT edge environment. 3. **Fusion label**: The fusion soft label is introduced, combined with the client's historical personalized knowledge, to guide the self - distillation process of the local model and realize the rapid review of historical personalized knowledge. Through extensive experimental verification, FedPSD significantly improves the model's personalized ability and overall performance on multiple Non - IID datasets without significantly increasing storage and computational overhead.

Federated Progressive Self-Distillation with Logits Calibration for Personalized IIoT Edge Intelligence

FediOS: Decoupling Orthogonal Subspaces for Personalization in Feature-skew Federated Learning

Personalized Edge Intelligence via Federated Self-Knowledge Distillation

Towards Personalized Federated Learning via Comprehensive Knowledge Distillation

Dual Calibration-based Personalised Federated Learning

DKD-pFed: A novel framework for personalized federated learning via decoupling knowledge distillation and feature decorrelation

PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning

FedD2S: Personalized Data-Free Federated Knowledge Distillation

Personalizing or Not: Dynamically Personalized Federated Learning with Incentives

FedCache: A Knowledge Cache-Driven Federated Learning Architecture for Personalized Edge Intelligence

A Personalized Federated Learning Method Based on Clustering and Knowledge Distillation

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

Spectral Co-Distillation for Personalized Federated Learning

Rethinking Client Drift in Federated Learning: A Logit Perspective

FedL2P: Federated Learning to Personalize

FedDWA: Personalized Federated Learning with Dynamic Weight Adjustment

pFedKT: Personalized federated learning with dual knowledge transfer

Personalized and privacy-enhanced federated learning framework via knowledge distillation

Towards Fairer and More Efficient Federated Learning via Multidimensional Personalized Edge Models

Improving Global Generalization and Local Personalization for Federated Learning

FedMeS: Personalized Federated Continual Learning Leveraging Local Memory