Constrained neighborhood preserving concept factorization for data representation

LuMei,ZhangLi,ZhaoXiang-Jun,LiFan-Zhang

2016-01-01

Abstract:Matrix factorization based techniques, such as nonnegative matrix factorization (NMF) and concept factorization (CF), have attracted a great deal of attentions in recent years, mainly due to their ...

What problem does this paper attempt to address?

Locality-constrained Concept Factorization

Haifeng Liu,Zheng Yang,Zhaohui Wu

DOI: https://doi.org/10.5591/978-1-57735-516-8/ijcai11-233

2011-01-01

Abstract:Matrix factorization based techniques, such as non-negative matrix factorization (NMF) and concept factorization (CF), have attracted great attention in dimension reduction and data clustering. Both of them are linear learning problems and lead to a sparse representation of the data. However, the sparsity obtained by these methods does not always satisfy locality conditions, thus the obtained data representation is not the best. This paper introduces a locality-constrained concept factorization method which imposes a locality constraint onto the traditional concept factorization. By requiring the concepts (basis vectors) to be as close to the original data points as possible, each data can be represented by a linear combination of only a few basis concepts. Thus our method is able to achieve sparsity and locality at the same time. We demonstrate the effectiveness of this novel algorithm through a set of evaluations on real world applications.
Discriminative Concept Factorization for Data Representation

Wei Hua,Xiaofei He

DOI: https://doi.org/10.1016/j.neucom.2011.07.020

IF: 6

2011-01-01

Neurocomputing

Abstract:Non-negative matrix factorization (NMF) has become a popular technique for finding low-dimensional representations of data. While the standard NMF can only be performed in the original feature space, one variant of NMF, named concept factorization, can be naturally kernelized and inherits all the strengths of NMF. To make use of label information, we propose a semi-supervised concept factorization technique called discriminative concept factorization (DCF) for data representation in this paper. DCF adopts a unified objective to combine the task of data reconstruction with the task of classification. These two tasks have mutual impacts on each other, which results in a concept factorization adapted to the classification task and a classifier built on the low-dimensional representations. Furthermore, we develop an iterative algorithm to solve the optimization problem through alternative convex programming. Experimental results on three real-word classification tasks demonstrate the effectiveness of DCF.
Constrained concept factorization for image representation.

Haifeng Liu,Genmao Yang,Zhaohui Wu,Deng Cai

DOI: https://doi.org/10.1109/TCYB.2013.2287103

2014-01-01

Abstract:Matrix factorization based techniques, such as nonnegative matrix factorization and concept factorization, have attracted great attention in dimensionality reduction and data clustering. Previous studies show that both of them yield impressive results on image processing and document clustering. However, both of them are essentially unsupervised methods and cannot incorporate label information. In this paper, we propose a novel semisupervised matrix decomposition method for extracting the image concepts that are consistent with the known label information. With this constraint, we call the new approach constrained concept factorization. By requiring that the data points sharing the same label have the same coordinate in the new representation space, this approach has more discriminating power. The experimental results on several corpora show good performance of our novel algorithm in terms of clustering accuracy and mutual information.
Local Coordinate Concept Factorization for Image Representation

Haifeng Liu,Zheng Yang,Ji Yang,Zhaohui Wu,Xuelong Li

DOI: https://doi.org/10.1109/tnnls.2013.2286093

IF: 14.255

2014-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:Learning sparse representation of high-dimensional data is a state-of-the-art method for modeling data. Matrix factorization-based techniques, such as nonnegative matrix factorization and concept factorization (CF), have shown great advantages in this area, especially useful for image representation. Both of them are linear learning problems and lead to a sparse representation of the images. However, the sparsity obtained by these methods does not always satisfy locality conditions. For example, the learned new basis vectors may be relatively far away from the original data. Thus, we may not be able to achieve the optimal performance when using the new representation for other learning tasks, such as classification and clustering. In this paper, we introduce a locality constraint into the traditional CF. By requiring the concepts (basis vectors) to be as close to the original data points as possible, each datum can be represented by a linear combination of only a few basis concepts. Thus, our method is able to achieve sparsity and locality simultaneously. We analyze the complexity of our novel algorithm and demonstrate the effectiveness in comparison with the state-of-the-art approaches through a set of evaluations based on real-world applications.
Neighborhood Preserving Nonnegative Matrix Factorization

Quanquan Gu,Jie Zhou

DOI: https://doi.org/10.5244/c.23.9

2009-01-01

Abstract:Nonnegative Matrix Factorization (NMF) has been widely used in computer vision and pattern recognition. It aims to find two nonnegative matrices whose product can well approximate the nonnegative data matrix, which naturally leads to parts-based and non-subtractive representation. In this paper, we present a neighborhood preserving nonnegative matrix factorization (NPNMF) for dimensionality reduction. It imposes an additional constraint on NMF that each data point can be represented as a linear combination of its neighbors. This constraint preserves the local geometric structure, and is good at dimensionality reduction on manifold. An iterative multiplicative updating algorithm is proposed to optimize the objective, and its convergence is guaranteed theoretically. Experiments on benchmark face recognition data sets demonstrate that the proposed method outperforms NMF as well as many state of the art dimensionality reduction methods.
Neighborhood Preserving Non-Negative Tensor Factorization For Image Representation

Yu-Xiong Wang,Liang-Yan Gui,Yu-Jin Zhang

DOI: https://doi.org/10.1109/ICASSP.2012.6288643

2012-01-01

ICASSP

Abstract:Non-negative Matrix Factorization (NMF) has become a powerful tool for image representation due to its enhanced semantic interpretability under non-negativity. Unfortunately, two types of neighborhood information essential to representation are lost in NMF. For individual image, the local structure information is missing in the vectorization, which can then be avoided by Non-negative Tensor Factorization (NTF). For image data points, they often reside on a low dimensional submanifold embedded in a high dimensional ambient space. NMF and NTF are incapable of encoding the local geometrical information, which can nevertheless be resuscitated by manifold learning. To simultaneously model both of the neighborhood relationship within and among image data, this paper proposes a novel algorithm called Neighborhood Preserving Non-negative Tensor Factorization (NPNTF) by incorporating locally linear embedding regularization into tensor factorization. Experimental results on image clustering show the superior performance of NPNTF with more natural and discriminating representation ability.
Robust Structured Nonnegative Matrix Factorization for Image Representation.

Zechao Li,Jinhui Tang,Xiaofei He

DOI: https://doi.org/10.1109/TNNLS.2017.2691725

IF: 14.255

2018-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:Dimensionality reduction has attracted increasing attention, because high-dimensional data have arisen naturally in numerous domains in recent years. As one popular dimensionality reduction method, nonnegative matrix factorization (NMF), whose goal is to learn parts-based representations, has been widely studied and applied to various applications. In contrast to the previous approaches, this pape...
Locality Preserving Nonnegative Matrix Factorization.

Deng Cai,Xiaofei He,Xuanhui Wang,Hujun Bao,Jiawei Han

2009-01-01

Abstract:Matrix factorization techniques have been frequently applied in information processing tasks. Among them, Non-negative Matrix Factorization (NMF) have received considerable attentions due to its psychological and physiological interpretation of naturally occurring data whose representation may be parts-based in human brain. On the other hand, from geometric perspective the data is usually sampled from a low dimensional manifold embedded in high dimensional ambient space. One hopes then to find a compact representation which uncovers the hidden topics and simultaneously respects the intrinsic geometric structure. In this paper, we propose a novel algorithm, called Locality Preserving Non-negative Matrix Factorization (LPNMF), for this purpose. For two data points, we use KL-divergence to evaluate their similarity on the hidden topics. The optimal maps are obtained such that the feature values on hidden topics are restricted to be non-negative and vary smoothly along the geodesics of the data manifold. Our empirical study shows the encouraging results of the proposed algorithm in comparisons to the state-of-the-art algorithms on two large high-dimensional databases.
Self-representative kernel concept factorization

Wenhui Wu,Yujie Chen,Ran Wang,Le Ou-Yang

DOI: https://doi.org/10.1016/j.knosys.2022.110051

2022-11-09

Abstract:Kernel concept factorization (KCF) has successfully utilized kernel trick to conduct matrix factorization in the kernel space. However, conventional KCF methods usually define kernel in advance, which limits their ability to exploit the power of kernel. This paper proposes a semi-supervised self-representative kernel concept factorization (S 3 RKCF) method to integrate adaptive kernel learning and low-dimensional data representation learning into a unified model. Technically, an adaptive local geometric structure is acquired in the KCF-induced self-representation space, and then it facilitates data representation learning simultaneously. Furthermore, to enhance the discriminability of data representation, limited supervisory information is imposed in the formulated optimization problem as constraints. In this way, our model can learn kernel and discriminative low-dimensional representation adaptively and iteratively. To solve the optimization problem, an alternating iterative algorithm is designed with convergence guarantee. The performance of our proposed S 3 RKCF is evaluated through clustering and classification tasks, and the experimental results on eight real-world data sets demonstrate its effectiveness compared to state-of-the-art methods.

computer science, artificial intelligence
Accelerating Locality Preserving Nonnegative Matrix Factorization

Guanhong Yao,Deng Cai

DOI: https://doi.org/10.1145/2396761.2398618

2012-01-01

Abstract:Matrix factorization techniques have been frequently applied in information retrieval, computer vision and pattern recognition. Among them, Non-negative Matrix Factorization (NMF) has received considerable attention due to its psychological and physiological interpretation of naturally occurring data whose representation may be parts-based in the human brain. Locality Preserving Non-negative Matrix Factorization (LPNMF) is a recently proposed graph-based NMF extension which tries to preserves the intrinsic geometric structure of the data. Compared with the original NMF, LPNMF has more discriminating power on data representa- tion thanks to its geometrical interpretation and outstanding ability to discover the hidden topics. However, the computa- tional complexity of LPNMF is O(n3), where n is the number of samples. In this paper, we propose a novel approach called Accelerated LPNMF (A-LPNMF) to solve the com- putational issue of LPNMF. Specifically, A-LPNMF selects p (p j n) landmark points from the data and represents all the samples as the sparse linear combination of these landmarks. The non-negative factors which incorporates the geometric structure can then be efficiently computed. Experimental results on the real data sets demonstrate the effectiveness and efficiency of our proposed method.
Non-negative Local Coordinate Factorization for Image Representation.

Yan Chen,Jiemi Zhang,Deng Cai,Wei Liu,Xiaofei He

DOI: https://doi.org/10.1109/tip.2012.2224357

2013-01-01

Abstract:Recently, nonnegative matrix factorization (NMF) has become increasingly popular for feature extraction in computer vision and pattern recognition. NMF seeks two nonnegative matrices whose product can best approximate the original matrix. The nonnegativity constraints lead to sparse parts-based representations that can be more robust than nonsparse global features. To obtain more accurate control over the sparseness, in this paper, we propose a novel method called nonnegative local coordinate factorization (NLCF) for feature extraction. NLCF adds a local coordinate constraint into the standard NMF objective function. Specifically, we require that the learned basis vectors be as close to the original data points as possible. In this way, each data point can be represented by a linear combination of only a few nearby basis vectors, which naturally leads to sparse representation. Extensive experimental results suggest that the proposed approach provides a better representation and achieves higher accuracy in image clustering.
Graph-based local concept coordinate factorization

Ping Li,Jiajun Bu,Lijun Zhang,Chun Chen

DOI: https://doi.org/10.1007/s10115-013-0715-x

2013-01-01

Abstract:Ubiquitous data are increasingly expanding in large volumes due to human activities, and grouping them into appropriate clusters is an important and yet challenging problem. Existing matrix factorization techniques have shown their significant power in solving this problem, e.g., nonnegative matrix factorization, concept factorization. Recently, one state-of-the-art method called locality-constrained concept factorization is put forward, but its locality constraint does not well reveal the intrinsic data structure since it only requires the concept to be as close to the original data points as possible. To address this issue, we present a graph-based local concept coordinate factorization (GLCF) method, which respects the intrinsic structure of the data through manifold kernel learning in the warped Reproducing Kernel Hilbert Space. Besides, a generalized update algorithm is developed to handle data matrices containing both positive and negative entries. Since GLCF is essentially based on the local coordinate coding and concept factorization, it inherits many advantageous properties, such as the locality and sparsity of the data representation. Moreover, it can better encode the locally geometrical structure via graph Laplacian in the manifold adaptive kernel. Therefore, a more compact and better structured representation can be obtained in the low-dimensional data space. Extensive experiments on several image and gene expression databases suggest the superiority of the proposed method in comparison with some alternatives.
Constrained Nonnegative Matrix Factorization for Image Representation

Haifeng Liu,Zhaohui Wu,Xuelong Li,Deng Cai,Thomas S. Huang

DOI: https://doi.org/10.1109/tpami.2011.217

IF: 23.6

2012-01-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:Nonnegative matrix factorization (NMF) is a popular technique for finding parts-based, linear representations of nonnegative data. It has been successfully applied in a wide range of applications such as pattern recognition, information retrieval, and computer vision. However, NMF is essentially an unsupervised method and cannot make use of label information. In this paper, we propose a novel semi-supervised matrix decomposition method, called Constrained Nonnegative Matrix Factorization (CNMF), which incorporates the label information as additional constraints. Specifically, we show how explicitly combining label information improves the discriminating power of the resulting matrix decomposition. We explore the proposed CNMF method with two cost function formulations and provide the corresponding update solutions for the optimization problems. Empirical experiments demonstrate the effectiveness of our novel algorithm in comparison to the state-of-the-art approaches through a set of evaluations based on real-world applications.
Robust and Discriminative Concept Factorization for Image Representation

Yuchen Guo,Guiguang Ding,Jile Zhou,Qiang Liu

DOI: https://doi.org/10.1145/2671188.2749317

2015-01-01

Abstract:Concept Factorization (CF), as a variant of Nonnegative Matrix Factorization (NMF), has been widely used for learning compact representation for images because of its psychological and physiological interpretation of naturally occurring data. And graph regularization has been incorporated into the objective function of CF to exploit the intrinsic low-dimensional manifold structure, leading to better performance. But some shortcomings are shared by existing CF methods. 1) The squared loss used to measure the data reconstruction quality is sensitive to noise in image data. 2) The graph regularization may lead to trivial solution and scale transfer problems for CF such that the learned representation is meaningless. 3) Existing methods mostly ignore the discriminative information in image data. In this paper, we propose a novel method, called Robust and Discriminative Concept Factorization (RDCF) for image representation. Specifically, RDCF explicitly considers the influence of noise by imposing a sparse error matrix, and exploits the discriminative information by approximate orthogonal constraints which can also lead to nontrivial solution. We propose an iterative multiplicative updating rule for the optimization of RDCF and prove the convergence. Experiments on 5 benchmark image datasets show that RDCF can significantly out-perform several state-of-the-art related methods, which validates the effectiveness of RDCF.
Non-Negative Matrix Factorization with Constraints.

Haifeng Liu,Zhaohui Wu

DOI: https://doi.org/10.1609/aaai.v24i1.7681

2010-01-01

Abstract:Non-negative matrix factorization (NMF), as a useful decomposition method for multivariate data, has been widely used in pattern recognition, information retrieval and computer vision. NMF is an effective algorithm to find the latent structure of the data and leads to a parts-based representation. However, NMF is essentially an unsupervised method and can not make use of label information. In this paper, we propose a novel semi-supervised matrix decomposition method, called Constrained Non-negative Matrix Factorization, which takes the label information as additional constraints. Specifically, we require that the data points sharing the same label have the same coordinate in the new representation space. This way, the learned representations can have more discriminating power. We demonstrate the effectiveness of this novel algorithm through a set of evaluations on real world applications.
General Subspace Constrained Non-Negative Matrix Factorization for Data Representation

Yong Liu,Yiyi Liao,Liang Tang,Feng Tang,Weicong Liu

DOI: https://doi.org/10.1016/j.neucom.2014.11.099

IF: 6

2015-01-01

Neurocomputing

Abstract:Nonnegative matrix factorization (NMF) has been proved to be a powerful data representation method, and has shown success in applications such as data representation and document clustering. However, the non-negative constraint alone is not able to capture the underlying properties of the data. In this paper, we present a framework to enforce general subspace constraints into NMF by augmenting the original objective function with two additional terms. One on constraints of the basis, the other on preserving the structural properties of the original data. This framework is general as it can be used to regularize NMF with a wide variety of subspace constraints that can be formulated into a certain form such as PCA, Fisher LDA and LPP. In addition, we present an iterative optimization algorithm to solve the general subspace constrained non-negative matrix factorization (GSC NMF). We show that the resulting subspace has enriched representation power as shown in our experiments.
Adaptive graph regularized nonnegative matrix factorization for data representation

Lin Zhang,Zhonghua Liu,Jiexin Pu,Bin Song

DOI: https://doi.org/10.1007/s10489-019-01539-9

IF: 5.3

2019-07-26

Applied Intelligence

Abstract:As a classical data representation method, nonnegative matrix factorization (NMF) can well capture the global structure information of the observed data, and it has been successfully applied in many fields. It is generally known that the local manifold structures will have a better effect than the global structures in image recognition and clustering. The local structure information can well be preserved by the neighbor graph in the manifold learning methods. The traditional neighbor graph constructed relies heavily on the original observed data. However, the original data usually contain a lot of noise and outliers in practical application, which results in obtaining inaccurate neighbor graph, and ultimately leads to performance degradation. How to get the ideal local structure information becomes more and more important. By combing the manifold learning into NMF, we propose an adaptive graph regularized nonnegative matrix factorization (AGNMF). In AGNMF, the neighbor graph is obtained by adaptive iteration. Both the global information and the local manifold can be well captured in AGNMF, and the better data representation can be obtained. A large number of experiments on different data sets show that our AGNMF has good clustering ability.

computer science, artificial intelligence
Non-negative Matrix Factorization on Manifold

Deng Cai,Xiaofei He,Xiaoyun Wu,Jiawei Han

DOI: https://doi.org/10.1109/ICDM.2008.57

2008-01-01

Abstract:Recently non-negative matrix factorization (NMF) has received a lot of attentions in information retrieval, computer vision and pattern recognition. NMF aims to find two non-negative matrices whose product can well approximate the original matrix. The sizes of these two matrices are usually smaller than the original matrix. This results in a compressed version of the original data matrix. The solution of NMF yields a natural parts-based representation for the data. When NMF is applied for data representation, a major disadvantage is that it fails to consider the geometric structure in the data. In this paper, we develop a graph based approach for parts-based data representation in order to overcome this limitation. We construct an affinity graph to encode the geometrical information and seek a matrix factorization which respects the graph structure. We demonstrate the success of this novel algorithm by applying it on real world problems.
Graph-Regularized Local Coordinate Concept Factorization for Image Representation

Jun Ye,Zhong Jin

DOI: https://doi.org/10.1007/s11063-017-9598-2

IF: 2.565

2017-01-01

Neural Processing Letters

Abstract:Existing matrix factorization based techniques, such as nonnegative matrix factorization and concept factorization, have been widely applied for data representation. In order to make the obtained concepts to be as close to the original data points as possible, one state-of-the-art method called locality constraint concept factorization is put forward, which represent the data by a linear combination of only a few nearby basis concepts. But its locality constraint does not well reveal the intrinsic data structure since it only requires the concept to be as close to the original data points as possible. To address these problems, by considering the manifold geometrical structure in local concept factorization via graph-based learning, we propose a novel algorithm, called graph-regularized local coordinate concept factorization (GRLCF). By constructing a parameter-free graph using constrained Laplacian rank (CLR) algorithm, we also present an extension of GRLCF algorithm as \(\hbox {GRLCF}_{\mathrm{CLR}}\). Moreover, we develop the iterative updating optimization schemes, and provide the convergence proof of our optimization scheme. Since GRLCF simultaneously considers the geometric structures of the data manifold and the locality conditions as additional constraints, it can obtain more compact and better structured data representation. Experimental results on ORL, Yale and Mnist image datasets demonstrate the effectiveness of our proposed algorithm.
Non-negative Matrix-Set Factorization

Le Li,Yu-Jin Zhang

DOI: https://doi.org/10.1109/icig.2007.103

2009-01-01

Abstract:Non-negative matrix factorization (NMF) is a recently developed, biologically inspired method for nonlinearly finding purely additive, sparse, linear, low-dimension representations of non-negative multivariate data to consequently make latent structure, feature or pattern in the data clear. Although it has been successfully applied in several research fields, it is confronted with three main problems, unsatisfactory accuracy, bad generality and high computational load, while the processed data appear as matrices. In this paper, a new method coined non-negative matrix-set factorization (NMSF) is developed to overcome the problems and an efficient, strictly monotonically convergent algorithm of NMSF is put forward. As opposed to NMF, NMSF directly processes original data matrices rather than vectorization results of them. Theoretical analysis and experimental results show that NMSF has higher accuracy, better generality and lower computational load than NMF.

Constrained neighborhood preserving concept factorization for data representation

Locality-constrained Concept Factorization

Discriminative Concept Factorization for Data Representation

Constrained concept factorization for image representation.

Local Coordinate Concept Factorization for Image Representation

Neighborhood Preserving Nonnegative Matrix Factorization

Neighborhood Preserving Non-Negative Tensor Factorization For Image Representation

Robust Structured Nonnegative Matrix Factorization for Image Representation.

Locality Preserving Nonnegative Matrix Factorization.

Self-representative kernel concept factorization

Accelerating Locality Preserving Nonnegative Matrix Factorization

Non-negative Local Coordinate Factorization for Image Representation.

Graph-based local concept coordinate factorization

Constrained Nonnegative Matrix Factorization for Image Representation

Robust and Discriminative Concept Factorization for Image Representation

Non-Negative Matrix Factorization with Constraints.

General Subspace Constrained Non-Negative Matrix Factorization for Data Representation

Adaptive graph regularized nonnegative matrix factorization for data representation

Non-negative Matrix Factorization on Manifold

Graph-Regularized Local Coordinate Concept Factorization for Image Representation

Non-negative Matrix-Set Factorization