Abstract:Semi-supervised learning (SSL) is a machine learning methodology that leverages unlabeled data in conjunction with a limited amount of labeled data. Although SSL has been applied in various applications and its effectiveness has been empirically demonstrated, it is still not fully understood when and why SSL performs well. Some existing theoretical studies have attempted to address this issue by modeling classification problems using the so-called Gaussian Mixture Model (GMM). These studies provide notable and insightful interpretations. However, their analyses are focused on specific purposes, and a thorough investigation of the properties of GMM in the context of SSL has been lacking. In this paper, we conduct such a detailed analysis of the properties of the high-dimensional GMM for binary classification in the SSL setting. To this end, we employ the approximate message passing and state evolution methods, which are widely used in high-dimensional settings and originate from statistical mechanics. We deal with two estimation approaches: the Bayesian one and the l2-regularized maximum likelihood estimation (RMLE). We conduct a comprehensive comparison between these two approaches, examining aspects such as the global phase diagram, estimation error for the parameters, and prediction error for the labels. A specific comparison is made between the Bayes-optimal (BO) estimator and RMLE, as the BO setting provides optimal estimation performance and is ideal as a benchmark. Our analysis shows that with appropriate regularizations, RMLE can achieve near-optimal performance in terms of both the estimation error and prediction error, especially when there is a large amount of unlabeled data. These results demonstrate that the l2 regularization term plays an effective role in estimation and prediction in SSL approaches.

Analysis of High-dimensional Gaussian Labeled-unlabeled Mixture Model via Message-passing Algorithm

Mixture model label propagation.

Mixture Distribution Modeling for Scalable Graph-Based Semi-Supervised Learning.

An Unsupervised Gaussian Mixture Classification Mechanism Based on Statistical Learning Analysis

A Semi-Supervised Learning Algorithm on Gaussian Mixture with Automatic Model Selection

Can semi-supervised learning use all the data effectively? A lower bound perspective

Semisupervised Bayesian Gaussian Mixture Models for Non-Gaussian Soft Sensor

Explicit Mutual Information Maximization for Self-Supervised Learning

Sample Complexity of Nonparametric Semi-Supervised Learning

The Supervised Learning Gaussian Mixture Model

Manifold Regularized Gaussian Mixture Model for Semi-supervised Clustering.

Semi‐supervised Gaussian mixture modelling with a missing‐data mechanism in R

Nonlinear Gaussian Mixture Regression for Multimode Quality Prediction with Partially Labeled Data.

Gaussian Message Passing for Overloaded Massive MIMO-NOMA

Semi-supervised Logistic Learning Based on Exponential Tilt Mixture Models

Semi-Supervised Sparse Gaussian Classification: Provable Benefits of Unlabeled Data

Probability-density-aware Semi-supervised Learning

Semi-Supervised Learning Through Label Propagation on Geodesics

Mixed Semi-Supervised Generalized-Linear-Regression with applications to Deep-Learning and Interpolators

A multi-manifold semi-supervised Gaussian mixture model for pattern classification

The High Separation Probability Assumption for Semi-Supervised Learning