Abstract:In this paper, we tackle the transductive semi-supervised learning problem that aims to obtain label predictions for the given unlabeled data points according to Vapnik's principle. Our proposed approach is based on optimal transport, a mathematical theory that has been successfully used to address various machine learning problems, and is starting to attract renewed interest in semi-supervised learning community. The proposed approach, Optimal Transport Propagation (OTP), performs in an incremental process, label propagation through the edges of a complete bipartite edge-weighted graph, whose affinity matrix is constructed from the optimal transport plan between empirical measures defined on labeled and unlabeled data. OTP ensures a high degree of predictions certitude by controlling the propagation process using a certainty score based on Shannon's entropy. We also provide a convergence analysis of our algorithm. Experiments task show the superiority of the proposed approach over the state-of-the-art. We make our code publicly available.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the label propagation problem in semi - supervised learning. Specifically, it aims to improve the label propagation method through the Optimal Transport (OT) theory. Traditional label propagation algorithms have some limitations when dealing with large - scale data, such as the need to add regularization terms to penalize mismatched predicted labels, difficulty in determining parameter values, and inability to capture the underlying geometric structure of the entire input space. ### Main Problem Description 1. **Limitations of Label Propagation Methods**: - Existing label propagation methods usually rely on fully - connected graphs or local neighborhood information, and these methods are not effective when dealing with large - scale data. - It is necessary to add regularization terms to penalize mismatched predicted labels, which increases the model complexity. - Parameter selection is difficult, for example, it is hard to determine the σ value in the Gaussian kernel. - It is unable to capture the underlying geometric structure of the entire input space, resulting in less accurate prediction results. 2. **Application of Optimal Transport Theory**: - The Optimal Transport theory provides a powerful tool for comparing probability measures and can capture the underlying geometric structure of data. - An improved similarity matrix can be constructed through the optimal transport plan, so as to perform label propagation better. 3. **Improving Prediction Confidence**: - The paper proposes a confidence scoring mechanism based on Shannon entropy to control the label propagation process and ensure the reliability of prediction results. ### Solutions The paper proposes a new label propagation method - Optimal Transport Propagation (OTP). The main innovations of this method include: - **Constructing a Complete Bipartite Weighted Graph**: A complete bipartite weighted graph is constructed through the optimal transport plan, eliminating the need to add regularization terms. - **Global Similarity Matrix**: Use the Optimal Transport theory to capture the global geometric structure of the entire input space, rather than just pairwise relationships or local neighborhood information. - **Incremental Label Propagation**: Adopt an incremental process to gradually propagate labels. In each iteration, newly labeled samples are added to the labeled set, thereby gradually improving the accuracy of prediction. - **Confidence Scoring**: Introduce a confidence scoring mechanism based on Shannon entropy to ensure the reliability of each predicted label. ### Experimental Verification The paper conducts experimental verification through seven benchmark datasets. The results show that the OTP method is superior to existing label propagation methods in multiple evaluation metrics, especially when the label noise assumption does not hold. In conclusion, this paper aims to solve the problems existing in existing label propagation methods by introducing the Optimal Transport theory and the confidence scoring mechanism, and provide a more efficient and reliable semi - supervised learning method.

Label Propagation Through Optimal Transport

Label Distribution Learning by Optimal Transport.

OT Cleaner: Label Correction As Optimal Transport

Incremental Confidence Sampling with Optimal Transport for Domain Adaptation

Learning to Count via Unbalanced Optimal Transport

OT-LLP: Optimal Transport for Learning from Label Proportions

OTMatch: Improving Semi-Supervised Learning with Optimal Transport

Inverse Entropic Optimal Transport Solves Semi-supervised Learning via Data Likelihood Maximization

Label Propagation Through Linear Neighborhoods

Label Propagation Via Teaching-to-Learn and Learning-to-Teach.

Joint Distribution Optimal Transportation for Domain Adaptation

Non-Metric Label Propagation

Semi-supervised Orthogonal Discriminant Analysis Via Label Propagation

Learning from Noisy Labels for Long-tailed Data via Optimal Transport

Recent Advances in Optimal Transport for Machine Learning

Optimal Transport for Generative Models

Relative Entropic Optimal Transport: a (Prior-Aware) Matching Perspective to (unbalanced) Classification

Hybrid Approach for Inductive Semi Supervised Learning using Label Propagation and Support Vector Machine

Metric Learning in Optimal Transport for Domain Adaptation

Linear Optimal Partial Transport Embedding

Semi-Supervised Optimal Transport for Heterogeneous Domain Adaptation