Abstract:In this work, we revisit the semi-supervised learning (SSL) problem from a new perspective of explicitly reducing empirical distribution mismatch between labeled and unlabeled samples. Benefited from this new perspective, we first propose a new deep semi-supervised learning framework called Semi-supervised Learning by Empirical Distribution Alignment (SLEDA), in which existing technologies from the domain adaptation community can be readily used to address the semi-supervised learning problem through reducing the empirical distribution distance between labeled and unlabeled data. Based on this framework, we also develop a new theoretical generalization bound for the research community to better understand the semi-supervised learning problem, in which we show the generalization error of semi-supervised learning can be effectively bounded by minimizing the training error on labeled data and the empirical distribution distance between labeled and unlabeled data. Building upon our new framework and the theoretical bound, we develop a simple and effective deep semi-supervised learning method called Augmented Distribution Alignment Network (ADA-Net) by simultaneously adopting the well-established adversarial training strategy from the domain adaptation community and a simple sample interpolation strategy for data augmentation. Additionally, we incorporate both strategies in our ADA-Net into two exiting SSL methods to further improve their generalization capability, which indicates that our new framework provides a complementary solution for solving the SSL problem. Our comprehensive experimental results on two benchmark datasets SVHN and CIFAR-10 for the semi-supervised image recognition task and another two benchmark datasets ModelNet40 and ShapeNet55 for the semi-supervised point cloud recognition task demonstrate the effectiveness of our proposed framework for SSL.

A Survey on Deep Semi-Supervised Learning

A survey on semi-supervised learning

An Overview of Deep Semi-Supervised Learning

Self-supervised Learning on Graphs: Contrastive, Generative,or Predictive

Semi-supervised Object Detection: A Survey on Recent Research and Progress

Semi-Supervised and Unsupervised Deep Visual Learning: A Survey

Recent Deep Semi-supervised Learning Approaches and Related Works

A Review of Semi Supervised Learning Theories and Recent Advances

Deep semi-supervised learning for medical image segmentation: A review

A Brief Survey on Semantic Segmentation with Deep Learning

Graph-based Semi-supervised Learning: A Comprehensive Review

Revisiting Deep Semi-supervised Learning: An Empirical Distribution Alignment Framework and Its Generalization Bound

A Survey towards Federated Semi-supervised Learning

Semi-supervised Deep Learning for Image Classification with Distribution Mismatch: A Survey

Small Data Challenges in Big Data Era: A Survey of Recent Progress on Unsupervised and Semi-Supervised Methods

A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends

Self-supervised Learning: Generative or Contrastive

A Survey on Label-efficient Deep Image Segmentation: Bridging the Gap between Weak Supervision and Dense Prediction

Safe semi-supervised learning: a brief introduction

Deep Semi-supervised Learning with Double-Contrast of Features and Semantics

A survey on Semi-, Self- and Unsupervised Learning for Image Classification