Abstract:Federated learning aims to tackle the ``isolated data island" problem, where it trains a collective model from physically isolated clients while safeguarding the privacy of users' data. However, supervised federated learning necessitates that each client labels their data for training, which can be both time-consuming and resource-intensive, and may even be impractical for edge devices. Moreover, the training and transmission of deep models present challenges to the computation and communication capabilities of the clients. To address these two inherent challenges in supervised federated learning, we propose a novel lightweight unsupervised federated learning approach that leverages unlabeled data on each client to perform lightweight model training and communication by harnessing pretrained vision-language models, such as CLIP. By capitalizing on the zero-shot prediction capability and the well-trained image encoder of the pre-trained CLIP model, we have carefully crafted an efficient and resilient self-training approach. This method refines the initial zero-shot predicted pseudo-labels of unlabeled instances through the sole training of a linear classifier on top of the fixed image encoder. Additionally, to address data heterogeneity within each client, we propose a class-balanced text feature sampling strategy for generating synthetic instances in the feature space to support local training. Experiments are conducted on multiple benchmark datasets. The experimental results demonstrate that our proposed method greatly enhances model performance in comparison to CLIP's zero-shot predictions and even outperforms supervised federated learning benchmark methods given limited computational and communication overhead.

Explaining Federated Learning Through Concepts in Image Classification.

Research on Image Classification Algorithm Based on Horizontal Federated Learning

Personalized Interpretation on Federated Learning: A Virtual Concepts approach

A Generalized Look at Federated Learning: Survey and Perspectives

Federated Machine Learning: Concept and Applications

Concept Matching: Clustering-based Federated Continual Learning

FedEmb: A Vertical and Hybrid Federated Learning Algorithm using Network And Feature Embedding Aggregation

MpFedcon : Model-Contrastive Personalized Federated Learning with the Class Center

Exploring Semantic Attributes from A Foundation Model for Federated Learning of Disjoint Label Spaces

Federated Learning in Practice: Reflections and Projections

Towards Client Driven Federated Learning

CLIP-guided Federated Learning on Heterogeneous and Long-Tailed Data

Completely Heterogeneous Federated Learning

Lightweight Unsupervised Federated Learning with Pretrained Vision Language Model

FedCCE: A class-level contribution explainable federated learning based on comparable prototypes collaboration for multi-site medical image classification.

Federated Learning from Pre-Trained Models: A Contrastive Learning Approach

Towards Few-Label Vertical Federated Learning

Federated Select: A Primitive for Communication- and Memory-Efficient Federated Learning

Rethinking Client Drift in Federated Learning: A Logit Perspective

Fuzzy-Based Concept Learning Method: Exploiting Data With Fuzzy Conceptual Clustering