Projective label propagation by label embedding

ZhangZhao,WangLei,JiaLei,LiFanzhang,ZhangLi,ZhaoMingbo

2017-01-01

Abstract:Virtually all existing label propagation (LP) approaches estimate the unknown labels of points from the original input space directly, but the transductive results are shallow and usually contain u...

What problem does this paper attempt to address?

Projective Label Propagation by Label Embedding: A Deep Label Prediction Framework for Representation and Classification

Zhao Zhang,Lei Wang,Lei Jia,Fanzhang Li,Li Zhang,Mingbo Zhao

DOI: https://doi.org/10.1016/j.knosys.2016.12.005

IF: 8.139

2016-01-01

Knowledge-Based Systems

Abstract:Virtually all existing label propagation (LP) approaches estimate the unknown labels of points from the original input space directly, but the transductive results are shallow and usually contain unfavorable mixed signs that may decrease the performance of both transductive models and out-of-sample extensions. To solve this issue, we propose a Projective Label Propagation (ProjLP) framework by label embedding, which can deliver more discriminating deep labels of samples to enhance representation and classification. ProjLP is originally proposed to improve the transductive prediction results by simultaneously learning a robust projection to remove the unfavorable mixed signs and convert the shallow labels into the discriminating deep ones. By including a regressive reconstruction loss term to correlate the extracted discriminative features with the deep labels, the out-of-sample extension of ProjLP is also presented by delivering a linear neighborhood preserving projection classifier, where the neighborhood preserving power is enabled by sharing the same graph weights in the original embedded deep label space and the approximated label space. Thus, the deep label of each new data can be obtained by embedding it directly onto the classifier. We also show the inclusion method by reconstructing the deep label of new data from the deep labels of its neighbors. To show the deep property of embedded deep labels, a multilayer network architecture of our model is illustrated. Simulations on artificial and real-world datasets demonstrate the validity of our techniques for representation and classification, compared with other state-of-the-arts.
Hypergraph Label Propagation Network.

Yubo Zhang,Nan Wang,Yufeng Chen,Changqing Zou,Hai Wan,Xinbin Zhao,Yue Gao

DOI: https://doi.org/10.1609/aaai.v34i04.6170

2020-01-01

Proceedings of the AAAI Conference on Artificial Intelligence

Abstract:In recent years, with the explosion of information on the Internet, there has been a large amount of data produced, and analyzing these data is useful and has been widely employed in real world applications. Since data labeling is costly, lots of research has focused on how to efficiently label data through semi-supervised learning. Among the methods, graph and hypergraph based label propagation algorithms have been a widely used method. However, traditional hypergraph learning methods may suffer from their high computational cost. In this paper, we propose a Hypergraph Label Propagation Network (HLPN) which combines hypergraph-based label propagation and deep neural networks in order to optimize the feature embedding for optimal hypergraph learning through an end-to-end architecture. The proposed method is more effective and also efficient for data labeling compared with traditional hypergraph learning methods. We verify the effectiveness of our proposed HLPN method on a real-world microblog dataset gathered from Sina Weibo. Experiments demonstrate that the proposed method can significantly outperform the state-of-the-art methods and alternative approaches.
Label Propagation Via Teaching-to-Learn and Learning-to-Teach.

Chen Gong,Dacheng Tao,Wei Liu,Liu,Jie Yang

DOI: https://doi.org/10.1109/tnnls.2016.2514360

IF: 14.255

2016-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:How to propagate label information from labeled examples to unlabeled examples over a graph has been intensively studied for a long time. Existing graph-based propagation algorithms usually treat unlabeled examples equally, and transmit seed labels to the unlabeled examples that are connected to the labeled examples in a neighborhood graph. However, such a popular propagation scheme is very likely to yield inaccurate propagation, because it falls short of tackling ambiguous but critical data points (e.g., outliers). To this end, this paper treats the unlabeled examples in different levels of difficulties by assessing their reliability and discriminability, and explicitly optimizes the propagation quality by manipulating the propagation sequence to move from simple to difficult examples. In particular, we propose a novel iterative label propagation algorithm in which each propagation alternates between two paradigms, teaching-to-learn and learning-to-teach (TLLT). In the teaching-to-learn step, the learner conducts the propagation on the simplest unlabeled examples designated by the teacher. In the learning-to-teach step, the teacher incorporates the learner’s feedback to adjust the choice of the subsequent simplest examples. The proposed TLLT strategy critically improves the accuracy of label propagation, making our algorithm substantially robust to the values of tuning parameters, such as the Gaussian kernel width used in graph construction. The merits of our algorithm are theoretically justified and empirically demonstrated through experiments performed on both synthetic and real-world data sets.
Bilinear Embedding Label Propagation: Towards Scalable Prediction of Image Labels.

Yuchen Liang,Zhao Zhang,Weiming Jiang,Mingbo Zhao,Fanzhang Li

DOI: https://doi.org/10.1109/lsp.2015.2488632

2015-01-01

IEEE Signal Processing Letters

Abstract:Traditional label propagation (LP) is shown to be effective for transductive classification. To enable the standard LP to handle outside images, two inductive methods by label reconstruction or by direct embedding have been presented, of which the latter scheme is relatively more efficient, especially for testing. But almost all inductive LP models use 1D vectors of images as inputs, which may destroy the topology structure of image pixels and usually suffer from high complexity due to the high dimension of 1D vectors in reality. To preserve the topology among pixels and address the scalability issue for the embedding based scheme, we propose a simple yet efficient Bilinear Embedding Label Propagation (BELP) by including a bilinear regularization term in terms of tensor representation to correlate the image labels with their bilinear features. BELP performs label prediction over the 2D matrices rather than 1D vectors, since images are essentially matrices. Finally, labels of new images can be easily obtained by embedding them onto a spanned bilinear subspace solved from a joint framework. Simulations verified the efficiency of our approach.
Label Propagation Through Linear Neighborhoods

Fei Wang,Changshui Zhang

DOI: https://doi.org/10.1109/tkde.2007.190672

2006-01-01

Abstract:In many practical data mining applications such as text classification, unlabeled training examples are readily available, but labeled ones are fairly expensive to obtain. Therefore, semi supervised learning algorithms have aroused considerable interests from the data mining and machine learning fields. In recent years, graph-based semi supervised learning has been becoming one of the most active research areas in the semi supervised learning community. In this paper, a novel graph-based semi supervised learning approach is proposed based on a linear neighborhood model, which assumes that each data point can be linearly reconstructed from its neighborhood. Our algorithm, named linear neighborhood propagation (LNP), can propagate the labels from the labeled points to the whole data set using these linear neighborhoods with sufficient smoothness. A theoretical analysis of the properties of LNP is presented in this paper. Furthermore, we also derive an easy way to extend LNP to out-of-sample data. Promising experimental results are presented for synthetic data, digit, and text classification tasks.
Kernel-Induced Label Propagation by Mapping for Semi-Supervised Classification

Zhao Zhang,Lei Jia,Mingbo Zhao,Guangcan Liu,Meng Wang,Shuicheng Yan

DOI: https://doi.org/10.1109/tbdata.2018.2797977

2018-01-01

IEEE Transactions on Big Data

Abstract:Kernel methods have been successfully applied to the areas of pattern recognition and data mining. In this paper, we mainly discuss the issue of propagating labels in kernel space. A Kernel-Induced Label Propagation (Kernel-LP) framework by mapping is proposed for high-dimensional data classification using the most informative patterns of data in kernel space. The essence of Kernel-LP is to perform joint label propagation and adaptive weight learning in a transformed kernel space. That is, our Kernel-LP changes the task of label propagation from the commonly-used Euclidean space in most existing work to kernel space. The motivation of our Kernel-LP to propagate labels and learn the adaptive weights jointly by the assumption of an inner product space of inputs, i.e., the original linearly inseparable inputs may be mapped to be separable in kernel space. Kernel-LP is based on existing positive and negative LP model, i.e., the effects of negative label information are integrated to improve the label prediction power. Also, Kernel-LP performs adaptive weight construction over the same kernel space, so it can avoid the tricky process of choosing the optimal neighborhood size suffered in traditional criteria. Two novel and efficient out-of-sample approaches for our Kernel-LP to involve new test data are also presented, i.e., (1) direct kernel mapping and (2) kernel mapping-induced label reconstruction, both of which purely depend on the kernel matrix between training set and testing set. Owing to the kernel trick, our algorithms will be applicable to handle the high-dimensional real data. Extensive results on real datasets demonstrate the effectiveness of our approach.
Semi-supervised label distribution learning via projection graph embedding

Xiuyi Jia,Tao Wen,Weiping Ding,Huaxiong Li,Weiwei Li

DOI: https://doi.org/10.1016/j.ins.2021.10.009

IF: 8.1

2021-12-01

Information Sciences

Abstract:Label distribution learning (LDL) is a new machine learning paradigm that addresses label ambiguity by emphasizing the relevance of each label to a particular instance. In supervised learning, many LDL algorithms have been proposed, which often require a large amount of well-annotated training data to achieve good performance. However, annotating a label distribution is more complicated and expensive than annotating a single label or multiple labels with logical values of 0 and 1. Thus, we propose a projection graph embedding algorithm for semi-supervised label distribution learning (PGE-SLDL). Specifically, we seek a potential space by orthogonal neighborhood preserving projections, named capture space. This capture space is used to select more valuable features and construct a graph that contains more accurate data structure information. We utilize the sample correlation information contained between graph nodes to recover the unknown label distribution. In addition, compared with fixed graphs in traditional semi-supervised learning, we carry out projection and graph construction simultaneously to obtain a self-updating projection graph, which is more helpful to learn label distribution. The experimental results validate the effectiveness of the proposed algorithm.

computer science, information systems
LSSLP – Local structure sensitive label propagation

Zhenfeng Zhu,Jian Cheng,Yao Zhao,Jieping Ye

DOI: https://doi.org/10.1016/j.ins.2015.11.007

IF: 8.1

2016-01-01

Information Sciences

Abstract:Label propagation is an approach to iteratively spread the prior state of label confidence associated with each of samples to its neighbors until achieving a global convergence state. Such process has been shown to hold close connection with a general graph-based regularization framework. Within this framework, a closed- form linear system can be built to carry out label propagation. In this paper, to address several issues inherent with previous graph-based label propagation framework, we propose a reformulated one, i.e., local structure sensitive label propagation (LSSLP). By associating each graph vertex with a local structure sensitive tuning factor, the empirical loss error on each vertex can be controlled preferably to keep consistent with the commonly preconditioned 'cluster assumption' of data structure. Out of consideration for information conservation, we relax the state conservation constraint of label confidence from labeled samples proposed by Belkin et¿al. (2004) to a more general form. Meanwhile, an inverse-warping procedure is incorporated into the proposed local structure sensitive label propagation framework to maintain large and stable enough classification margin. Based on the felicitous inversion technique for blocked matrix, we extend LSSLP to its incremental and inductive versions and also present computationally efficient implementation of it. Experimental results demonstrate the performance of the reformulated regularization framework for label propagation is much competitive.
Label Propagation Through Sparse Neighborhood and Its Applications

Fei Zang,Jiang-She Zhang

DOI: https://doi.org/10.1016/j.neucom.2012.03.017

IF: 6

2012-01-01

Neurocomputing

Abstract:In this paper, a novel semi-supervised learning approach is proposed. It assumes that, for the ith sample x"i, the samples from x"i's sparse neighborhood have the same label with x"i and the label of x"i can be linearly reconstructed by the labels of those samples from x"i's sparse neighborhood. Our algorithm firstly selects the sparse neighborhood for each sample, and then in that sparse neighborhood finds the sparse coefficients to represent the local geometry structure, finally seeks a label propagation way. Different from many existing methods, we construct the adapting graph, simultaneously, give the weight of each edge. What's more, we highlight the role of those samples in that sparse neighborhood, meanwhile, eliminate the role of those samples out of that sparse neighborhood. The experimental results on face recognition and document classification demonstrate the effectiveness and efficiency of our proposed approach in this paper.
Two-Stage Label Embedding Via Neural Factorization Machine for Multi-Label Classification

Chen,Haobo Wang,Weiwei Liu,Xingyuan Zhao,Tianlei Hu,Gang Chen

DOI: https://doi.org/10.1609/aaai.v33i01.33013304

2019-01-01

Abstract:Label embedding has been widely used as a method to exploit label dependency with dimension reduction in multilabel classification tasks. However, existing embedding methods intend to extract label correlations directly, and thus they might be easily trapped by complex label hierarchies. To tackle this issue, we propose a novel Two-Stage Label Embedding (TSLE) paradigm that involves Neural Factorization Machine (NFM) to jointly project features and labels into a latent space. In encoding phase, we introduce a Twin Encoding Network (TEN) that digs out pairwise feature and label interactions in the first stage and then efficiently learn higher-order correlations with deep neural networks (DNNs) in the second stage. After the codewords are obtained, a set of hidden layers is applied to recover the output labels in decoding phase. Moreover, we develop a novel learning model by leveraging a max margin encoding loss and a label-correlation aware decoding loss, and we adopt the mini-batch Adam to optimize our learning model. Lastly, we also provide a kernel insight to better understand our proposed TSLE. Extensive experiments on various real-world datasets demonstrate that our proposed model significantly outperforms other state-of-the-art approaches.
Transformed Neighborhood Propagation

Zhao Zhang,Fan-Zhang Li,Ming-Bo Zhao

DOI: https://doi.org/10.1109/ICPR.2014.651

2014-01-01

Abstract:An enhanced label propagation technique termed transformed neighborhood propagation (TNP) is proposed for semi-supervised learning. In the TNP setting, the processes of constructing weighted similarity graph and propagating label information of the labeled data to unlabeled points are conducted in the transformed feature space. TNP is mainly motivated by a fact that the optimal feature representation Y with possible unfavorable features and noises in the original data X removed by feature learning are more appropriate and accurate for measuring pair wise similarities of samples. To achieve the representation Y, The recent marginal semi-supervised sub-manifold projections is applied, so enhanced inter-class separation and enhanced intra-class compactness are delivered at the same time. The similarity graph is finally constructed based on Y. We also propose to calculate semi-supervised reconstruction weights for the weight assignment. As a result, the label estimation power can be enhanced by benefiting from the refined weighted similarity graph over Y instead of X, through propagating the labels of points in the transformed space for prediction. Visualization and image classification verified the effectiveness of our TNP, compared with other related label propagation algorithms.
ResLPA: Label Propagation with Residual Learning.

Yi Luo,Rui Huang,Aiguo Chen,Xi Zeng

DOI: https://doi.org/10.1145/3467707.3467752

2021-01-01

Abstract:Label Propagation Algorithm (LPA) predicts what label a node in a graph has by its accumulated label information after iteratively propagating label information among adjacent nodes. Subsequent studies of LPA try to assign weights on connections to adjust the propagation between different node pairs. However, such approaches still rely on the assumption that adjacent nodes should have similar labels. To predict when the label distribution may differ from this assumption, we propose Residual Label Propagation (ResLPA). It approximates residuals between labels with node features, then adds the approximated residuals to the label information as corrections before propagating. In experiments on five real-world graphs, ResLPA matches or exceeds state-of-the-art LPA-based methods, GCN-based methods and their integrations.
Multi-Label Learning Based on Iterative Label Propagation over Graph

Bin Fu,Zhihai Wang,Guandong Xu,Longbing Cao

DOI: https://doi.org/10.1016/j.patrec.2014.01.001

IF: 4.757

2014-01-01

Pattern Recognition Letters

Abstract:One key challenge in multi-label learning is how to exploit label dependency effectively, and existing methods mainly address this issue via training a prediction model for each label based on the combination of original features and the labels on which it depends on. However, the influence of label dependency might be depressed due to the significant imbalance in dimensionality of feature set and dependent label set in this way, also the dynamic interaction between labels cannot be utilized effectively. In this paper, we propose a new framework to exploit the dependencies between labels iteratively and interactively. Every label's prediction will be updated through iterative process of propagation, other than being determined directly by a prediction model. Specifically, we utilize a graph model to encode the dependencies between labels, and employ the random-walk with restart (RWR) strategy to propagate the dependency among all labels iteratively until the predictions for all the labels converge. We validate our approach by experiments, and the results demonstrate that it yields significant improvements compared with several state-of-the-art algorithms. (C) 2014 Published by Elsevier B.V.
Teaching-To-Learn And Learning-To-Teach For Multi-Label Propagation

Chen Gong,Dacheng Tao,Jie Yang,Wei Liu

DOI: https://doi.org/10.1609/aaai.v30i1.10191

2016-01-01

Abstract:Multi-label propagation aims to transmit the multi-label information from labeled examples to unlabeled examples based on a weighted graph. Existing methods ignore the specific propagation difficulty of different unlabeled examples and conduct the propagation in an imperfect sequence, leading to the error-prone classification of some difficult examples with uncertain labels. To address this problem, this paper associates each possible label with a "teacher", and proposes a "Multi-Label Teaching-to-Learn and Learning-toTeach" (ML-TLLT) algorithm, so that the entire propagation process is guided by the teachers and manipulated from simple examples to more difficult ones. In the teaching-to-learn step, the teachers select the simplest examples for the current propagation by investigating both the definitiveness of each possible label of the unlabeled examples, and the dependencies between labels revealed by the labeled examples. In the learning-to-teach step, the teachers reversely learn from the learner's feedback to properly select the simplest examples for the next propagation. Thorough empirical studies show that due to the optimized propagation sequence designed by the teachers, ML-TLLT yields generally better performance than seven state-of-the-art methods on the typical multi-label benchmark datasets.
Label propagation based supervised locality projection analysis for plant leaf classification

Shanwen Zhang,Yingke Lei,Tianbao Dong,Xiao-Ping Zhang

DOI: https://doi.org/10.1016/j.patcog.2013.01.015

IF: 8

2013-01-01

Pattern Recognition

Abstract:The label propagation has the benefits of nearly-linear running time and easy implementation. In this paper, we make use of the label propagation to propose a new weight measure, and present a supervised locality projection analysis (SLPA) method for plant leaf classification. Firstly, we apply Warshall algorithm to label propagation and get the label matrix, then incorporate it into the weight, which has a clear physical meaning. Secondly, multi-class data points in high-dimensional space are to be pulled or pushed by discriminant neighbors to form an optimum projecting to low dimensionality. Finally, the experimental results on two plant leaf databases show that the proposed method is quite effective and feasible.
3D Model Labeling Method Based on Multi-Label Propagation

Hengke DING,Zizhao WU,Yigang WANG

DOI: https://doi.org/10.3778/j.issn.1002-8331.1807-0071

2019-01-01

Abstract:With the progress of data acquisition equipment and modeling technology, how to efficiently analyze and retrieve 3D shapes has become a hot topic in the field of geometric processing. Currently, many works are focused on the classification of models, almost only consider single label. When dealing with the multi-label problem, the correlation between tags and samples is ignored ,as a result, a lot of computation time is spent. To address this problem, a 3d model labeling method based on multi-label propagation is proposed. The core of this approach is to explore the potential of multi-label correlation in the whole sample space by using the label and sample correlation. In particular, a small set of multi-labeled samples are given, it iterates dynamic metric fusion with label information and the label information of these samples are propagated to the unlabeled samples. The process of propagation depends on similarity matrices between samples and the relationship between labels and samples is taken into account. The labeling results are gotten until the iteration finish. Experimental results on the PSB dataset demonstrates the high accuracy of the approach with a few interactions.
SPL-LDP: a Label Distribution Propagation Method for Semi-Supervised Partial Label Learning

Moxian Song,Chenxi Sun,Derun Cai,Shenda Hong,Hongyan Li

DOI: https://doi.org/10.1007/s10489-023-04548-x

IF: 5.3

2023-01-01

Applied Intelligence

Abstract:Partial label learning learns from examples represented by a single instance while associated with multiple candidate labels, among which only one valid label resides. However, in real-world applications, collecting candidate label sets for all training examples is costly. As unlabeled data are being considered as a indispensable ingredient for low-cost computing, the semi-supervised partial label learning underlying propagating labels between partially labeled and unlabeled instances has grown progressively momentous. Nevertheless, the noisy information carried by false-positive instances hides in the candidate label sets and is propagated as well. In this work, we propose a label distribution propagation based approach, namely Spl-ldp, which can jointly learn from partially labeled and unlabeled instances. Specifically, the label distribution of partially labeled instances is mined based on the topological information. Instead of directly logic label propagation, an iterative label distribution propagation procedure between partially labeled and unlabeled instances is subsequently employed to leverage the data distribution of unlabeled instances. Unseen instances are classified with the minimum reconstruction error on the whole data sets. Extensive experiments on five real-world data sets show that the proposed Spl-ldp method performs favorably against baselines.
A Label Propagation Approach Based on Local Optimization

Xiaohong Zhang,Yulin Jiang,Jianji Ren,Chaosheng Tang

DOI: https://doi.org/10.1142/s012918311850047x

2018-01-01

International Journal of Modern Physics C

Abstract:Community detection offers an important way to understand the structures and functions of social network. The label propagation algorithm has attracted vast attention since it is very suitable for discovering communities from large-scale networks. However, the algorithm suffers from the instability and inefficiency problem caused by the random policies it adopted. In this paper, we propose a novel label propagation approach based on local optimization to deal with the problem. The approach introduces a pre-propagation mechanism to optimize randomly initialized labels according to special factors, for example, node compactness. After that, it traverses and relabels nodes in the descending order of aggregate influence. The experiment results demonstrate the usefulness and effectiveness of our approach.
Non-Metric Label Propagation

Yin Zhang,Zhi-hua Zhou

2009-01-01

Abstract:In many applications non-metric distances are bet- ter than metric distances in reflecting the percep- tual distances of human beings. Previous studies on non-metric distances mainly focused on super- vised setting and did not consider the usefulness of unlabeled data. In this paper, we present probably the first study of label propagation on graphs in- duced from non-metric distances. The challenge here lies in the fact that the triangular inequality does not hold for non-metric distances and there- fore, a direct application of existing label propa- gation methods will lead to inconsistency and con- flict. We show that by applying spectrum transfor- mation, non-metric distances can be converted into metric ones, and thus label propagation can be ex- ecuted. Such methods, however, suffer from the change of original semantic relations. As a main result of this paper, we prove that any non-metric distance matrix can be decomposed into two met- ric distance matrices containing different informa- tion of the data. Based on this recognition, our pro- posed NMLP method derives two graphs from the original non-metric distance and performs a joint label propagation on the joint graph. Experiments validate the effectiveness of the proposed NMLP method.

Projective label propagation by label embedding

Projective Label Propagation by Label Embedding: A Deep Label Prediction Framework for Representation and Classification

Hypergraph Label Propagation Network.

Label Propagation Via Teaching-to-Learn and Learning-to-Teach.

Bilinear Embedding Label Propagation: Towards Scalable Prediction of Image Labels.

Label Propagation Through Linear Neighborhoods

Kernel-Induced Label Propagation by Mapping for Semi-Supervised Classification

Semi-supervised label distribution learning via projection graph embedding

LSSLP – Local structure sensitive label propagation

Label Propagation Through Sparse Neighborhood and Its Applications

Two-Stage Label Embedding Via Neural Factorization Machine for Multi-Label Classification

Transformed Neighborhood Propagation

ResLPA: Label Propagation with Residual Learning.

Multi-Label Learning Based on Iterative Label Propagation over Graph

Teaching-To-Learn And Learning-To-Teach For Multi-Label Propagation

Label propagation based supervised locality projection analysis for plant leaf classification

3D Model Labeling Method Based on Multi-Label Propagation

SPL-LDP: a Label Distribution Propagation Method for Semi-Supervised Partial Label Learning

A Label Propagation Approach Based on Local Optimization

Non-Metric Label Propagation