Abstract:Contrastive learning has emerged as a promising method for addressing the non-independent and identically distributed (non-IID) problem in federated learning. Existing methods make the limited assumption that all clients are consistently available in each communication round. However, in the more constrained scenario of cross-device settings, resource-constrained clients intermittently participate in the training process, resulting in the updates of the local models being delayed and thus reducing the representation consistency in the federated contrastive loss. In this paper, we analyse the superiority of the federated contrastive loss over traditional methods in terms of addressing the non-IID problem: the federated contrastive loss not only corrects the local objective towards the global objective but also debiases the local updates. To address the representation inconsistency issue, we propose a novel method called Federated Similarity-aware Exponential Moving Average update (FedSEMA), which incorporates a similarity-aware function into the EMA update process. First, FedSEMA adaptively facilitates the underlying pairwise collaborations between clients to generate personalized knowledge based on the similarity-aware EMA update procedure. Second, FedSEMA effectively exploits personalized knowledge to update the delayed local models, maintaining representation consistency to maximally benefit representation learning. Our extensive experiments conducted on various datasets under different non-IID settings demonstrate that FedSEMA is an effective and robust method for tackling the representation inconsistency issue.

FedSEMA: Similarity-Aware for Representation Consistency in Federated Contrastive Learning

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

Federated Self-Supervised Contrastive Learning via Ensemble Similarity Distillation

FedSiam-DA: Dual-aggregated Federated Learning Via Siamese Network for Non-Iid Data

FedSSA: Semantic Similarity-based Aggregation for Efficient Model-Heterogeneous Personalized Federated Learning

AsyncFedED: Asynchronous Federated Learning with Euclidean Distance Based Adaptive Weight Aggregation

Towards Communication-Efficient and Privacy-Preserving Federated Representation Learning

FedSiam-DA: Dual-aggregated Federated Learning via Siamese Network under Non-IID Data

Relaxed Contrastive Learning for Federated Learning

Federated Contrastive Learning for Personalized Semantic Communication

FedSSC: Shared Supervised-Contrastive Federated Learning

Tackling Feature Skew in Heterogeneous Federated Learning with Semantic Enhancement

Balancing Similarity and Complementarity for Federated Learning

Model-Contrastive Federated Learning

FedSeg: Class-Heterogeneous Federated Learning for Semantic Segmentation

FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Update

Beyond Similarity: Personalized Federated Recommendation with Composite Aggregation

Aiding Global Convergence in Federated Learning via Local Perturbation and Mutual Similarity Information

Enhancing Edge-Assisted Federated Learning with Asynchronous Aggregation and Cluster Pairing