Abstract:Federated learning (FL) enables independent parties to collaboratively build machine learning (ML) models while protecting data privacy. Vertical federated learning (VFL), a variant of FL, has recently drawn increasing attention as the VFL matches the enterprises' demands of leveraging more valuable features to achieve better model performance without jeopardizing data privacy. However, conventional VFL may run into data deficiency as it is only able to exploit aligned samples (belonging to different parties) with labels, leaving often the majority of unaligned and unlabeled samples unused. The data deficiency hampers the effort of the federation. In this work, we propose a Federated Hybrid Self-Supervised Learning framework, coined FedHSSL, to utilize all available data (including unaligned and unlabeled samples) of participants to train the joint VFL model. The core idea of FedHSSL is to utilize cross-party views (i.e., dispersed features) of samples aligned among parties and local views (i.e., augmentations) of samples within each party to improve the representation learning capability of the joint VFL model through SSL (e.g., SimSiam). FedHSSL further exploits generic features shared among parties to boost the performance of the joint model through partial model aggregation. We empirically demonstrate that our FedHSSL achieves significant performance gains compared with baseline methods, especially when the number of labeled samples is small. We provide an in-depth analysis of FedHSSL regarding privacy leakage, which is rarely discussed in existing self-supervised VFL works. We investigate the protection mechanism for FedHSSL. The results show our protection can thwart the state-of-the-art label inference attack.

Hybrid Federated Learning: Algorithms and Implementation

Hybrid Federated Learning: Algorithms and Implementation.

Hybrid Federated Learning for Feature & Sample Heterogeneity: Algorithms and Implementation

FedEmb: A Vertical and Hybrid Federated Learning Algorithm using Network And Feature Embedding Aggregation

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

A Generalized Look at Federated Learning: Survey and Perspectives

Completely Heterogeneous Federated Learning

Federated mutual learning

FedH2L: A Federated Learning Approach with Model and Statistical Heterogeneity

Effective and Efficient Federated Tree Learning on Hybrid Data

Accelerating Hybrid Federated Learning Convergence under Partial Participation

Heterogeneous Federated Learning with Cross-layer Model Fusion.

Vertical Federated Learning Hybrid Local Pre-training

A Hybrid Self-Supervised Learning Framework for Vertical Federated Learning

Federated Mutual Learning: a Collaborative Machine Learning Method for Heterogeneous Data, Models, and Objectives

How global observation works in Federated Learning: Integrating vertical training into Horizontal Federated Learning

Clustered Federated Learning in Heterogeneous Environment.

Advancements in Federated Learning: Models, Methods, and Privacy

A Survey of Federated Learning on Non-IID Data

FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation

Federated Learning under Partially Class-Disjoint Data via Manifold Reshaping