Abstract:Federated Long-Tailed Learning (Fed-LT), a paradigm wherein data collected from decentralized local clients manifests a globally prevalent long-tailed distribution, has garnered considerable attention in recent times. In the context of Fed-LT, existing works have predominantly centered on addressing the data imbalance issue to enhance the efficacy of the generic global model while neglecting the performance at the local level. In contrast, conventional Personalized Federated Learning (pFL) techniques are primarily devised to optimize personalized local models under the presumption of a balanced global data distribution. This paper introduces an approach termed Federated Local and Generic Model Training in Fed-LT (FedLoGe), which enhances both local and generic model performance through the integration of representation learning and classifier alignment within a neural collapse framework. Our investigation reveals the feasibility of employing a shared backbone as a foundational framework for capturing overarching global trends, while concurrently employing individualized classifiers to encapsulate distinct refinements stemming from each client's local features. Building upon this discovery, we establish the Static Sparse Equiangular Tight Frame Classifier (SSE-C), inspired by neural collapse principles that naturally prune extraneous noisy features and foster the acquisition of potent data representations. Furthermore, leveraging insights from imbalance neural collapse's classifier norm patterns, we develop Global and Local Adaptive Feature Realignment (GLA-FR) via an auxiliary global classifier and personalized Euclidean norm transfer to align global features with client preferences. Extensive experimental results on CIFAR-10/100-LT, ImageNet, and iNaturalist demonstrate the advantage of our method over state-of-the-art pFL and Fed-LT approaches.

Federated Learning With Long-Tailed Data via Representation Unification and Classifier Rectification

FedCRAC: Improving Federated Classification Performance on Long-Tailed Data Via Classifier Representation Adjustment and Calibration

Integrating Local Real Data with Global Gradient Prototypes for Classifier Re-Balancing in Federated Long-Tailed Learning

FedLF: Adaptive Logit Adjustment and Feature Optimization in Federated Long-Tailed Learning

Rethinking the Representation in Federated Unsupervised Learning with Non-IID Data

Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Features

Federated deep long-tailed learning: A survey

No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifier

Federated learning on non-IID and globally long-tailed data via meta re-weighting networks

Decoupling Representation and Classifier for Long-Tailed Recognition

FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Data

Towards Federated Long-Tailed Learning

Forgetting Through Transforming: Enabling Federated Unlearning via Class-Aware Representation Transformation

Federated unsupervised representation learning

Federated Learning with Bilateral Curation for Partially Class-Disjoint Data

Federated Learning under Partially Class-Disjoint Data via Manifold Reshaping

Addressing Skewed Heterogeneity via Federated Prototype Rectification with Personalization

FedBnR: Mitigating federated learning Non-IID problem by breaking the skewed task and reconstructing representation

Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data

FedRAD: Heterogeneous Federated Learning via Relational Adaptive Distillation

FedCRL: Personalized Federated Learning with Contrastive Shared Representations for Label Heterogeneity in Non-IID Data