Abstract:Self-supervised representation learning and deep clustering are mutually beneficial to learn high-quality representations and cluster data simultaneously in centralized settings. However, it is not always feasible to gather large amounts of data at a central entity, considering data privacy requirements and computational resources. Federated Learning (FL) has been developed successfully to aggregate a global model while training on distributed local data, respecting the data privacy of edge devices. However, most FL research effort focuses on supervised learning algorithms. A fully unsupervised federated clustering scheme has not been considered in the existing literature. We present federated momentum contrastive clustering (FedMCC), a generic federated clustering framework that can not only cluster data automatically but also extract discriminative representations training from distributed local data over multiple users. In FedMCC, we demonstrate a two-stage federated learning paradigm where the first stage aims to learn differentiable instance embeddings and the second stage accounts for clustering data automatically. The experimental results show that FedMCC not only achieves superior clustering performance but also outperforms several existing federated self-supervised methods for linear evaluation and semi-supervised learning tasks. Additionally, FedMCC can easily be adapted to ordinary centralized clustering through what we call momentum contrastive clustering (MCC). We show that MCC achieves state-of-the-art clustering accuracy results in certain datasets such as STL-10 and ImageNet-10. We also present a method to reduce the memory footprint of our clustering schemes.

FedCo: Self-Supervised Learning in Federated Learning with Momentum Contrast.

FedDGP: Disentangling Global and Personal Models for Federated Learning

Federated Momentum Contrastive Clustering

FedMoS: Taming Client Drift in Federated Learning with Double Momentum and Adaptive Selection.

FedCOA: an Efficient Cluster Optimization for Federated Learning

FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction

FedCos: A Scene-Adaptive Enhancement for Federated Learning

Fedcm: Federated learning with client-level momentum

FedCos: A Scene-adaptive Federated Optimization Enhancement for Performance Improvement

Momentum Benefits Non-iid Federated Learning Simply and Provably

Federated Learning via Consensus Mechanism on Heterogeneous Data: A New Perspective on Convergence

FedEntropy: Information-entropy-aided Training Optimization of Semi-Supervised Federated Learning

Fake It Till Make It: Federated Learning with Consensus-Oriented Generation

Efficient Federated Learning Using Layer-Wise Regulation and Momentum Aggregation*

FedProc: Prototypical contrastive federated learning on Non-IID data

FedCML: Federated Clustering Mutual Learning with non-IID Data.

CoFED: Cross-silo Heterogeneous Federated Multi-task Learning via Co-training

Lazy Aggregation for Heterogeneous Federated Learning

FedCore: Straggler-Free Federated Learning with Distributed Coresets

FedGroup: Efficient Clustered Federated Learning via Decomposed Data-Driven Measure