Abstract:In this work, we revisit the semi-supervised learning (SSL) problem from a new perspective of explicitly reducing empirical distribution mismatch between labeled and unlabeled samples. Benefited from this new perspective, we first propose a new deep semi-supervised learning framework called Semi-supervised Learning by Empirical Distribution Alignment (SLEDA), in which existing technologies from the domain adaptation community can be readily used to address the semi-supervised learning problem through reducing the empirical distribution distance between labeled and unlabeled data. Based on this framework, we also develop a new theoretical generalization bound for the research community to better understand the semi-supervised learning problem, in which we show the generalization error of semi-supervised learning can be effectively bounded by minimizing the training error on labeled data and the empirical distribution distance between labeled and unlabeled data. Building upon our new framework and the theoretical bound, we develop a simple and effective deep semi-supervised learning method called Augmented Distribution Alignment Network (ADA-Net) by simultaneously adopting the well-established adversarial training strategy from the domain adaptation community and a simple sample interpolation strategy for data augmentation. Additionally, we incorporate both strategies in our ADA-Net into two exiting SSL methods to further improve their generalization capability, which indicates that our new framework provides a complementary solution for solving the SSL problem. Our comprehensive experimental results on two benchmark datasets SVHN and CIFAR-10 for the semi-supervised image recognition task and another two benchmark datasets ModelNet40 and ShapeNet55 for the semi-supervised point cloud recognition task demonstrate the effectiveness of our proposed framework for SSL.

Open-Set Semi-Supervised Learning by Distribution Alignment

An Empirical Study and Analysis on Open-Set Semi-Supervised Learning

Exploration and Exploitation of Unlabeled Data for Open-Set Semi-Supervised Learning

Label Distribution-based Open-world Semi-supervised Learning

Multi-Task Curriculum Framework for Open-Set Semi-Supervised Learning

Revisiting Deep Semi-supervised Learning: An Empirical Distribution Alignment Framework and Its Generalization Bound

RDA: Reciprocal Distribution Alignment for Robust Semi-supervised Learning

Open-Set Semi-Supervised Learning for 3D Point Cloud Understanding

Open-Domain Semi-Supervised Learning via Glocal Cluster Structure Exploitation

Open World Semi-supervised Learning Based on Multi-scale Enhanced Feature

SCOMatch: Alleviating Overtrusting in Open-set Semi-supervised Learning

RDA: Reciprocal Distribution Alignment for Robust Semi-supervised Learning

DC-SSL: Addressing Mismatched Class Distribution in Semi-supervised Learning

Robust Semi-supervised Learning by Wisely Leveraging Open-set Data

Rethinking Open-World Semi-Supervised Learning: Distribution Mismatch and Inductive Inference

OCI-SSL: Open Class-Imbalanced Semi-Supervised Learning with Contrastive Learning

Trash to Treasure: Harvesting OOD Data with Cross-Modal Matching for Open-Set Semi-Supervised Learning

Partial Optimal Transport Based Out-of-Distribution Detection for Open-Set Semi-Supervised Learning

LaRW: Boosting Open-Set Semi-Supervised Learning with Label-Guided Re-Weighting

Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning

A Benchmark on Robust Semi-Supervised Learning in Open Environments