Abstract:Federated learning is a distributed paradigm that allows multiple parties to collaboratively train deep learning models without direct exchange of raw data. Nevertheless, the inherent non-independent and identically distributed (non-i.i.d.) nature of data distribution among clients results in significant degradation of the acquired model. The primary goal of this study is to develop a robust federated learning algorithm to address feature shift in clients' samples, potentially arising from a range of factors such as acquisition discrepancies in medical imaging. To reach this goal, we first propose federated feature augmentation ( FedFA $^{l}$ ), a novel feature augmentation technique tailored for federated learning. FedFA $^{l}$ is based on a crucial insight that each client's data distribution can be characterized by first-/second-order statistics ( a.k.a. , mean and standard deviation) of latent features; and it is feasible to manipulate these local statistics globally , i.e. , based on information in the entire federation, to let clients have a better sense of the global distribution across clients. Grounded on this insight, we propose to augment each local feature statistic based on a normal distribution, wherein the mean corresponds to the original statistic, and the variance defines the augmentation scope. Central to FedFA $^{l}$ is the determination of a meaningful Gaussian variance, which is accomplished by taking into account not only biased data of each individual client, but also underlying feature statistics represented by all participating clients. Beyond consideration of low-order statistics in FedFA $^{l}$ , we propose a federated feature alignment component ( FedFA $^{h}$ ) that exploits higher-order feature statistics to gain a more detailed understanding of local feature distribution and enables explicit alignment of augmented features in different clients to promote more consistent feature learning. Combining FedFA $^{l}$ and FedFA $^{h}$ yields our full approach FedFA$+$ . FedFA$+$ is non-parametric, incurs negligible additional communication costs, and can be seamlessly incorporated into popular CNN and Transformer architectures. We offer rigorous theoretical analysis as well as extensive empirical justifications to demonstrate the effectiveness of the algorithm. Our implementation is publicly available at https://github.com/tfzhou/FedFA .

FedAA: Using Non-sensitive Modalities to Improve Federated Learning while Preserving Image Privacy

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

FedDAA: a robust federated learning framework to protect privacy and defend against adversarial attack

FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning

FedFC: an Efficient Personalized Federated Learning Method on Non-iid Data

FedGA: A Greedy Approach to Enhance Federated Learning with Non-IID Data

Dual Adversarial Federated Learning on Non-IID Data

Data Augmentation Scheme for Federated Learning with Non-Iid Data

Federated Learning with Data-Agnostic Distribution Fusion

Federated Feature Augmentation and Alignment

FedFed: Feature Distillation Against Data Heterogeneity in Federated Learning

FedSea: Federated Learning Via Selective Feature Alignment for Non-IID Multimodal Data

APPFed: A Hybrid Privacy-Preserving Framework for Federated Learning over Sensitive Data

Personalized Federated Learning with Adaptive Feature Extraction and Category Prediction in Non-IID Datasets

Federated Learning Model Training Method Based on Data Features Perception Aggregation

Federated mutual learning

FedALA: Adaptive Local Aggregation for Personalized Federated Learning

FedFA: Federated Feature Augmentation

Fedadkd:heterogeneous federated learning via adaptive knowledge distillation

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

Neural-aware Decoupling Fusion Based Personalized Federated Learning for Intelligent Sensing