Abstract:Semi-supervised learning (SSL) is an important theme in machine learning, in which we have a few labeled samples and many unlabeled samples. In this paper, for SSL in a regression problem, we consider a method of incorporating information on unlabeled samples into kernel functions. As a typical implementation, we employ Gaussian kernels whose centers are labeled and unlabeled input samples. Since the number of coefficients is larger than the number of labeled samples in this setting, this is an over-parameterized regression roblem. A ridge regression is a typical estimation method under this setting. In this paper, alternatively, we consider to apply the minimum norm least squares (MNLS), which is known as a helpful tool for understanding deep learning behavior while it may not be application oriented. Then, in applying the MNLS for SSL, we established several methods based on feature extraction/dimension reduction in the SVD (singular value decomposition) representation of a Gram type matrix appeared in the over-parameterized regression problem. The methods are thresholding according to singular value magnitude with cross validation, hard-thresholding with cross validation, universal thresholding and bridge thresholding methods. The first one is equivalent to a method using a well-known low rank approximation of a Gram type matrix. We refer to these methods as SVD regression methods. In the experiments for real data, depending on datasets, clear superiority of the proposed SVD regression methods over ridge regression methods was observed. And, depending on datasets, incorporation of information on unlabeled input samples into kernels was found to be clearly effective.

Semi-supervised kernel regression

Semi-Supervised Kernel Regression

A Semi-supervised Kernel Learning Method Based on Label Propagation

Semi-supervised Contrastive Regression for Pharmaceutical Processes

Distributed Semi-Supervised Learning with Kernel Ridge Regression

Semi-supervised regression via embedding space mapping and pseudo-label smearing

Semi-Supervised Graph Embedding Via Multi-instance Kernel Learning

Spectral Transformation Approaches to Semi-supervised Learning

Soft Sensors of Nonlinear Industrial Processes Based on Self-Learning Kernel Regression Model

Semi-supervised Image Classification Via Nonnegative Least-Squares Regression

A semi-supervised learning using over-parameterized regression

Semi-supervised Regression with Data Partitioning and Feature Mapping

Maximum Margin Based Semi-supervised Spectral Kernel Learning.

Semi-supervised regression with label-guided adaptive graph optimization

Semi-supervised Kernel Density Estimation for Video Annotation

Adaptive Semi-Supervised Learning with Discriminative Least Squares Regression

Spectral Kernel Learning for Semi-Supervised Classification.

Semi-Supervised Hybrid Local Kernel Regression for Soft Sensor Modelling of Rubber-Mixing Process

Semi-Supervised Regression using Spectral Techniques

Semi-Supervised Feature Selection Via Sparse Rescaled Linear Square Regression.

Semi-supervised learning with extremely sparse labeled data on multiple semi-supervised assumptions