Abstract:In order to address the problem of data heterogeneity, in recent years, personalized federated learning has tailored models to individual user data to enhance model performance on clients with diverse data distributions. However, the existing personalized federated learning methods do not adequately address the problem of data heterogeneity, and lack the processing of system heterogeneity. Consequently, these issues lead to diminished training efficiency and suboptimal model performance of personalized federated learning in heterogeneous environments. In response to these challenges, we propose FedPRL, a novel approach to personalized federated learning designed specifically for heterogeneous environments. Our method tackles data heterogeneity by implementing a personalized strategy centered on local data storage, enabling the accurate extraction of features tailored to the data distribution of individual clients. This personalized approach enhances the performance of federated learning models when dealing with non-IID data. To overcome system heterogeneity, we design a client selection mechanism grounded in reinforcement learning and user quality evaluation. This mechanism optimizes the selection of clients based on data quality and training time, thereby boosting the efficiency of the training process and elevating the overall performance of personalized models. Moreover, we devise a local training method that utilizes global knowledge distillation of non-target classes, which combined with traditional federated learning can effectively address the issue of catastrophic forgetting during global model updates. This approach enhances the generalization capability of the global model and further improves the performance of personalized models. Extensive experiments on both standard and real-world datasets demonstrate that FedPRL effectively resolves the challenges of data and system heterogeneity, enhancing the efficiency and model performance of personalized federated learning methods in heterogeneous environments, and outperforming state-of-the-art methods in terms of model accuracy and training efficiency.

Data Quality Aware Hierarchical Federated Reinforcement Learning Framework for Dynamic Treatment Regimes

A Federated Learning Framework Via Decentralized Data Valuation for Chronic Disease Healthcare

Efficient Ring-Topology Decentralized Federated Learning with Deep Generative Models for Medical Data in Ehealthcare Systems

Deep Reinforcement Learning for Dynamic Treatment Regimes on Medical Registry Data

Federated Inverse Reinforcement Learning for Smart ICUs with Differential Privacy

Kernel-Based Distributed Q-Learning: A Scalable Reinforcement Learning Approach for Dynamic Treatment Regimes

Dynamic Fair Federated Learning Based on Reinforcement Learning

Embracing Multiheterogeneity and Privacy Security Simultaneously: A Dynamic Privacy-Aware Federated Reinforcement Learning Approach

Adversarial reinforcement learning for dynamic treatment regimes

Reward Shaping Based Federated Reinforcement Learning

Communication-Efficient Hybrid Federated Learning for E-health with Horizontal and Vertical Data Partitioning

An Efficient Personalized Federated Learning Approach in Heterogeneous Environments: a Reinforcement Learning Perspective

Optimizing Hierarchical Federated Learning: A Reinforcement Learning Approach

Personalized Retrogress-Resilient Framework for Real-World Medical Federated Learning

Data Quality Control in Federated Instruction-tuning of Large Language Models

Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation

Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination

Towards Dynamic Resource Allocation and Client Scheduling in Hierarchical Federated Learning: A Two-Phase Deep Reinforcement Learning Approach

Hierarchical Reinforcement Learning for Automatic Disease Diagnosis

Privacy-Preserving Heterogeneous Federated Learning for Sensitive Healthcare Data

Deep reinforcement learning for multi-class imbalanced training: applications in healthcare