Constrained discriminant neighborhood embedding for high dimensional data feature extraction

LeiLei,ZhangXiao-Ping

IF: 6

2016-01-01

Neurocomputing

Abstract:When handling pattern classification problem such as face recognition and digital handwriting identification, image data is always represented to high dimensional vectors, from which discriminant f...

What problem does this paper attempt to address?

Constrained discriminant neighborhood embedding for high dimensional data feature extraction.

Bo Li,Lei Lei,Xiao-Ping Zhang

DOI: https://doi.org/10.1016/j.neucom.2015.01.099

IF: 6

2016-01-01

Neurocomputing

Abstract:When handling pattern classification problem such as face recognition and digital handwriting identification, image data is always represented to high dimensional vectors, from which discriminant features are extracted using dimensionality reduction methods. So in this paper, we present a supervised manifold learning based dimensionality reduction method named constrained discirminant neighborhood embedding (CDNE). In the proposed CDNE, on one hand, the class information of samples is taken into account to construct both an inter-class graph and an intra-class graph, where neighborhood points in the intra-class graph are selected from those with the same class and any point in the inter-class graph should sample those labeled different classes as its neighborhood points. On the other hand, locally least linear reconstruction technique is also introduced to model an objective function under the local uncorrelation constraint to explore a discriminant subspace. Compared to some related and state-of-the-art dimensionality reduction methods such as discriminant neighborhood embedding (DNE), supervised locality discriminant manifold learning (SLDML), discriminant sparse neighborhood preserving embedding (DSNPE), local graph embedding based on maximum margin criterion (LGE/MMC), uncorrelated discriminant locality preserving projection (UDLPP) and locally uncorrelated discriminant projection (LUDP), the proposed CDNE has been validated to be efficient and feasible by experimental results on some benchmark face data sets including CMU PIE, ORL and FERET.
Locality Pursuit Embedding

WL Min,K Lu,XF He

DOI: https://doi.org/10.1016/j.patcog.2003.09.005

IF: 8

2004-01-01

Pattern Recognition

Abstract:Dimensionality reduction techniques are widespread in pattern recognition research. Principal component analysis, as one of the most popular methods used, is optimal when the data points reside on a linear subspace. Nevertheless, it may fail to preserve the local structure if the data reside on some nonlinear manifold, which is indisputably important in many real applications, especially when nearest-neighbor search is involved. In this paper, we propose locality pursuit embedding, a linear algorithm that arises by solving a variational problem. It produces a linear embedding that respects the local geometrical structure described by the Euclidean distances. Some illustrative examples are presented along with applications to real data sets.
Combination of Local Multiple Patterns and Exponential Discriminant Analysis for facial recognition

Lifang Zhou,Bin Fang,Weisheng Li,Lidou Wang

2013-01-01

Abstract:Global features-based methods and local features -based methods have been very successful in face recognition system, yet they can be combined together and jointly optimized so as to minimize the error of a nearest-neighbor classifier. We consider both descriptor for face images with Local Multiple Pattern, and discriminant learning techniques with Exponential Discriminant Analysis. A combination framework based on Local Multiple Pattern and Exponential Discriminant Analysis has been proposed in this paper. Firstly, our approach encodes the multi-scale face feature by Local Multiple Pattern, and then they have been extended to strengthen the discriminative ability by Exponential Discriminant Analysis; Secondly, we suggest to use the above feature on different layers independently so that a multiple classifier system can be attained. Using these techniques, we obtain the state-of-the-art performance on two public available databases. © 2013 IFSA.
Discriminant Neighborhood Embedding for Classification.

Wei Zhang,Xiangyang Xue,Hong Lu,Yue-Fei Guo

DOI: https://doi.org/10.1016/j.patcog.2006.05.011

IF: 8

2006-01-01

Pattern Recognition

Abstract:In this paper a novel subspace learning method called discriminant neighborhood embedding (DNE) is proposed for pattern classification. We suppose that multi-class data points in high-dimensional space tend to move due to local intra-class attraction or inter-class repulsion and the optimal embedding from the point of view of classification is discovered consequently. After being embedded into a low-dimensional subspace, data points in the same class form compact submanifod whereas the gaps between submanifolds corresponding to different classes become wider than before. Experiments on the UMIST and MNIST databases demonstrate the effectiveness of our method.
Constrained Linear Discrimination Analysis and Face Recognition

LI Xiao-run,Guangzhou Zhao,ZHAO Liao-ying

2008-01-01

Abstract:A constrained linear discrimination analysis method was proposed for the feature extraction in the pattern recognition of problems with high dimension and small samples. Applying whitening process and Gram-Schimdt orthogonalization and orthogonal subspace projection, an optimal transformation matrix was designed to minimize the ratio of intra-class distance to inter-class distance while imposing the constraint that different class centers after transformation are along specifically directions that are orthogonal each other. For the small sample problem of face recognition, the whitening process was realized by singular value decomposition. The experimental results using the ORL and the UMIST face image database demonstrate that the effectiveness and performance of CLDA is approximate with some Fisher discrimination analysis such as direct Fisher discrimination analysis (DDA) and regularized Fisher discriminant analysis (R-LDA).
Face Feature Extraction and Recognition Via Local Binary Pattern and Two-Dimensional Locality Preserving Projection

Lijian Zhou,Hui Wang,Wanquan Liu,Zhe-Ming Lu

DOI: https://doi.org/10.1007/s11042-018-6868-6

IF: 2.577

2018-01-01

Multimedia Tools and Applications

Abstract:In this paper, we propose a novel face feature extraction approach based on Local Binary Pattern (LBP) and Two Dimensional Locality Preserving Projections (2DLPP) to enhance the texture features and preserve the space structure properties of a face image. LBP is firstly used to remove the effect of illumination and noise, which would enhance the detailed texture characteristics of face images. Then 2DLPP is performed to extract some prominent features and decrease the image dimension with space structure information. The Nearest Neighborhood Classifier (NNC) is used to recognize a face image at the end. In addition, the rule for dimension selection is studied from the results of experiments about choosing an appropriate feature dimension by 2DLPP computation. The experimental results on the Yale, the extended Yale B and CMU PIE C09 benchmark datasets showed that the proposed face feature extraction and recognition method achieves a better performance in comparison with similar techniques, and the proposed dimension selection rule can give an appropriate feature dimension in 2DLPP.
Two Dimensional Discriminant Neighborhood Preserving Embedding in Face Recognition

Meng Pang,Jifeng Jiang,Chuang Lin,Binghui Wang

DOI: https://doi.org/10.1117/12.2178728

2015-01-01

Abstract:One of the key issues of face recognition is to extract the features of face images. In this paper, we propose a novel method, named two-dimensional discriminant neighborhood preserving embedding (2DDNPE), for image feature extraction and face recognition. 2DDNPE benefits from four techniques, i.e., neighborhood preserving embedding (NPE), locality preserving projection (LPP), image based projection and Fisher criterion. Firstly, NPE and LPP are two popular manifold learning techniques which can optimally preserve the local geometry structures of the original samples from different angles. Secondly, image based projection enables us to directly extract the optimal projection vectors from two-dimensional image matrices rather than vectors, which avoids the small sample size problem as well as reserves useful structural information embedded in the original images. Finally, the Fisher criterion applied in 2DDNPE can boost face recognition rates by minimizing the within-class distance, while maximizing the between-class distance. To evaluate the performance of 2DDNPE, several experiments are conducted on the ORL and Yale face datasets. The results corroborate that 2DDNPE outperforms the existing 1D feature extraction methods, such as NPE, LPP, LDA and PCA across all experiments with respect to recognition rate and training time. 2DDNPE also delivers consistently promising results compared with other competing 2D methods such as 2DNPP, 2DLPP, 2DLDA and 2DPCA.
Fuzzy local discriminant embedding for image feature extraction

Pu Huang,Zhangjing Yang,Caikou Chen

DOI: https://doi.org/10.1016/j.compeleceng.2015.03.013

IF: 4.152

2015-01-01

Computers & Electrical Engineering

Abstract:In pattern recognition, feature extraction techniques have been widely employed to reduce the high dimensionality of data. In this paper, we propose a novel algorithm called fuzzy local discriminant embedding (FLDE) based on the unsupervised discriminant projection criterion and fuzzy set theory for image feature extraction and recognition. In the proposed method, a membership degree matrix is firstly calculated using the fuzzy k-nearest neighbor (FKNN) algorithm, and then the membership degree and the label information are incorporated into the definition of the weighted matrices to get the fuzzy local scatter and fuzzy nonlocal scatter. After characterizing the fuzzy nonlocal scatter and the fuzzy local scatter, a concise feature extraction criterion is derived via maximizing the ratio between them. Experimental results on the ORL, FERET, and CMU PIE face databases show the effectiveness of the proposed method.
Face Recognition Using Data Driven Local Appearance Features

Xin Xie,Chao Chen,Zhijian Chen

DOI: https://doi.org/10.2991/icmmcce-15.2015.514

2015-01-01

Abstract:A novel data driven face descriptor based on point-to-subspace metric is proposed for subspace classifiers. Unlike conventional feature descriptors which are carefully designed by hand, the newly proposed method uses supervised learning to derive more robust and more discriminative descriptors. During the feature extraction process, the point-to-subspace distance is used as the inner mechanism to train parameters including filters and weights of different pixels. Experimental results on FERET and Extended Yale B database show that when using subspace classifiers, the proposed feature descriptor is more discriminative and yields higher recognition rate over other features.
Neighbourhood Sensitive Preserving Embedding for Pattern Classification.

Bing-Hui Wang,Chuang Lin,Xue-Feng Zhao,Zhe-Ming Lu

DOI: https://doi.org/10.1049/iet-ipr.2013.0539

IF: 2.3

2014-01-01

IET Image Processing

Abstract:Recently, a large family of supervised or unsupervised manifold learning algorithms that stem from statistical or geometrical theory has been designed to solve the problem of pattern classification. In this study, consider the fact that the data are usually sampled from a low-dimensional manifold space which resides in a high-dimensional Euclidean space, the authors propose a novel two-graph-based supervised linear classification algorithm called neighbourhood sensitive preserving embedding (NSPE). Different from local linear embedding (LLE) (or neighbourhood preserving embedding (NPE)) which preserves the local neighbourhood structure with one graph, NSPE can discover both the intrinsic and discriminant structure of the data manifold by constructing two graphs, that is, the within-class graph and the between-class graph. Thus, the data are mapped into a subspace where the nearby points with the same label are close to each other, whereas the nearby points with different labels are far apart. As a classification method, besides being defined on training samples, NSPE is also defined on testing samples. Experiments carried on the real-world face databases demonstrate that the results of all two-graph-based spectral methods are comparable and better than that of one-graph-based methods.
Data-Dependent Kernel Discriminant Analysis for Feature Extraction and Classification

Jun-Bao Li,Jeng-Shyang Pan,Zhe-Ming Lu,Bin-Yih Liao

DOI: https://doi.org/10.1109/icia.2006.305931

2006-01-01

Abstract:Subspace analysis is an effective technique for feature extraction, which aims at finding a low-dimensional space of high-dimensional data. In this paper, a novel subspace analysis method based on Data-Dependent Kernel Discriminant Analysis (DDKDA) is proposed for dimension reduction. The procedure of DDKDA contains two stages: one is to find the optimal combination coefficients by solving a constrained optimization function which transformed to an eigenvalue problem; other is to implement KDA under the optimal data-dependent kernel with Fisher criterion. DDKDA is more adaptive to the input data than KDA owing to the optimization of projection from input space to feature space with the data-dependent kernel, which enhances the performance of KDA. Experiments on the ORL and Yale face databases demonstrate the good performance of the proposed algorithm.
By Normalizing To Improve Generalized Foley-Sammon Transform In High-Dimensional Spaces-----With Application To Face Recognition

Guang Dai,Yuntao Qian

DOI: https://doi.org/10.1109/ICSMC.2004.1400650

2004-01-01

Abstract:Linear discriminant analysis (LDA) is an effective feature extraction technique for classification. A new LDA-based algorithm, i.e., direct normalized generalized Foley-Sammon transform (DN-GFST) method in high dimensional spaces, is presented in this paper. It not only overcomes the limitation of traditional LDA that they overemphasize the larger distance between classes and cause large overlaps of neighboring classes, but also has the best separable ability in global sense. Lastly, our method is applied to facial image recognition, and the experimental results show that the performance of the present method is superior to those of the existing methods in terms of the classification error rate.
Enhanced Local Gradient Order Features and Discriminant Analysis for Face Recognition

Chuan-Xian Ren,Zhen Lei,Dao-Qing Dai,Stan Z Li

DOI: https://doi.org/10.1109/TCYB.2015.2484356

Abstract:Robust descriptor-based subspace learning with complex data is an active topic in pattern analysis and machine intelligence. A few researches concentrate the optimal design on feature representation and metric learning. However, traditionally used features of single-type, e.g., image gradient orientations (IGOs), are deficient to characterize the complete variations in robust and discriminant subspace learning. Meanwhile, discontinuity in edge alignment and feature match are not been carefully treated in the literature. In this paper, local order constrained IGOs are exploited to generate robust features. As the difference-based filters explicitly consider the local contrasts within neighboring pixel points, the proposed features enhance the local textures and the order-based coding ability, thus discover intrinsic structure of facial images further. The multimodal features are automatically fused in the most discriminant subspace. The utilization of adaptive interaction function suppresses outliers in each dimension for robust similarity measurement and discriminant analysis. The sparsity-driven regression model is modified to adapt the classification issue of the compact feature representation. Extensive experiments are conducted by using some benchmark face data sets, e.g., of controlled and uncontrolled environments, to evaluate our new algorithm.
Efficient Feature Extraction For Image Classification

Wei Zhang,Xiangyang Xue,Ziehen Sun,Yue-fei Guo,Mingmin Chi,Hong Lu

DOI: https://doi.org/10.1109/ICCV.2007.4409058

2007-01-01

Abstract:In many image classification applications, input feature space is often high-dimensional and dimensionality reduction is necessary to alleviate the curse of dimensionality or to reduce the cost of computation. In this paper, we extract discriminant features for image classification by learning a low-dimensional embedding from finite labeled samples. In the new feature space, intra-class compactness and extra-class separability are achieved simultaneously. Target dimensionality of the embedding is selected by spectral analysis. Our method is designed suitable for data with both uni- and multi-modal class distributions. We also develop its two-dimensional variant which makes use of the matrix representation of images. Experimental results on three real image datasets demonstrate the efficacy of our method compared to the state of the art.
Local Feature Discriminant Projection

Mengyang Yu,Ling Shao,Xiantong Zhen,Xiaofei He

DOI: https://doi.org/10.1109/tpami.2015.2497686

IF: 23.6

2016-01-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:In this paper, we propose a novel subspace learning algorithm called Local Feature Discriminant Projection (LFDP) for supervised dimensionality reduction of local features. LFDP is able to efficiently seek a subspace to improve the discriminability of local features for classification. We make three novel contributions. First, the proposed LFDP is a general supervised subspace learning algorithm which provides an efficient way for dimensionality reduction of large-scale local feature descriptors. Second, we introduce the Differential Scatter Discriminant Criterion (DSDC) to the subspace learning of local feature descriptors which avoids the matrix singularity problem. Third, we propose a generalized orthogonalization method to impose on projections, leading to a more compact and less redundant subspace. Extensive experimental validation on three benchmark datasets including UIUC-Sports, Scene-15 and MIT Indoor demonstrates that the proposed LFDP outperforms other dimensionality reduction methods and achieves state-of-the-art performance for image classification.
Tied Factors Analysis for High-Dimensional Image Feature Extraction and Recognition Application

Haibin Liao,Youbin Chen,Wenhua Dai,Ruolin Ruan

DOI: https://doi.org/10.1007/s10044-016-0572-9

2016-01-01

Abstract:Feature extraction from images, which are typical of high dimensionality, is crucial to the recognition performance. To explore the discriminative information while depressing the intra-class variations due to variable illumination and view conditions, we propose a factor analysis framework for separate “content” from “style,” identifying a familiar face seen under unfamiliar viewing conditions, classifying familiar poses presented in an unfamiliar face, estimating age across unfamiliar faces. The framework applies efficient algorithms derived from objective factor separating functions and space mapping functions, which can produce sufficiently expressive representations of feature extraction and dimensionality reduction. We report promising results on three different tasks in the high-dimensional image perceptual domains: face identification with two benchmark face databases, facial pose classification with a benchmark facial pose database, extrapolation of age to unseen facial image. Experimental results show that our approach produced higher classification performance when compared to classical LDA, WLDA, LPP, MFA, and DLA algorithms.
Linear Laplacian Discrimination for Feature Extraction

Deli Zhao,Zhouchen Lin,Rong Xiao,Xiaoou Tang

DOI: https://doi.org/10.1109/CVPR.2007.383125

2007-01-01

Abstract:Discriminant feature extraction plays a fundamental role in pattern recognition. In this paper, we propose the linear Laplacian discrimination (LLD) algorithm/or discriminant feature extraction. LLD is an extension of linear discriminant analysis (LDA). Our motivation is to address the issue that LDA cannot work well in cases where sample spaces are non-Euclidean. Specifically, we define the within-class scatter and the between-class scatter using similarities which are based on pairwise distances in sample spaces. Thus the structural information of classes is contained in the within-class and the between-class Laplacian matrices which are free from metrics of sample spaces. The optimal discriminant subspace can be derived by controlling the structural evolution of Laplacian matrices. Experiments are performed on the facial database for FRGC version 2. Experimental results show that LLD is effective in extracting discriminant features.
Feature Extraction By Locality-Based Linear Discriminant Analysis

Pu Huang,Caikou Chen

2010-01-01

Abstract:In this paper, we present a novel feature extraction technique, called locality-based linear discriminant analysis (LLDA), for multi-class discriminant tasks. The proposed method can be regarded as an improved version of the classical linear discriminant analysis (LDA), which is one of the most popular feature extraction methods. LLDA has at least two distinct advantages compared with LDA. Firstly, LDA may fail to discover the potential structure in the data, while LLDA integrates both class label information and neighborhood relationships between samples, so LLDA has the ability to reveal the intrinsic structure in the data which is more helpful to solve such nonlinear problem as recognition tasks than LDA. Secondly, LDA usually can only extract at most C-1 (C is the number of sample classes) features in a real situation, which doesn't suffice to categorized all samples, while LLDA is able to obtain much more features. Therefore, LLDA has more powerful discriminant ability than LDA. We carry out LLDA and LDA, as well as other current popular algorithms such as PCA, LPP and MFA on the ORL face database and the CENPARPARMI handwritten numerical database, and the experimental results show that LLDA can achieve much higher recognition rate.
Locality-based Linear Discriminant Projection

Ding Xinlong,Huang Pu,Chen Caikou

2012-01-01

Abstract:This paper presents a novel feature extraction technique, called locality-based linear discriminant projection (LLDP), for multi-class discriminant tasks. The proposed method can be regarded as an improved version of the classical linear discriminant projection (LDA), which is one of the most popular feature extraction methods. LLDP has at least two distinct advantages compared with LDA. Firstly, LDA may fail to discover the potential structure in the data, while LLDP integrates both class label information and neighborhood relationships between samples, so LLDP has the ability to reveal the intrinsic structure in the data which is more helpful to solve such nonlinear problem as recognition tasks than LDA. Secondly, LDA usually can only extract at most C-1 (C is the number of sample classes) features in a real situation, which doesn't suffice to categorized all samples, while LLDP is able to obtain much more features. Therefore, LLDP has more powerful discriminant ability than LDA. We carry out LLDP and LDA, as well as other current popular algorithms such as PCA, LPP and MFA on the ORL face database and the CENPARPARMI handwritten numerical database, and the experimental results show that LLDP can achieve much higher recognition rate.
Supervised Neighborhood Preserving Embedding for Feature Extraction and Its Application for Soft Sensor Modeling

Xiaofeng Yuan,Zhiqiang Ge,Lingjian Ye,Zhihuan Song

DOI: https://doi.org/10.1002/cem.2811

IF: 2.5

2016-01-01

Journal of Chemometrics

Abstract:Neighborhood preserving embedding (NPE) is a useful tool for learning the manifold of high‐dimensional data. As a linear approximation of nonlinear locally linear embedding, NPE can be applied to dimensionality reduction by neighborhood preserving. However, the original NPE algorithm is an unsupervised method, which extracts features without any reference to the output information. In this paper, a supervised NPE framework is proposed for output‐related feature extraction in soft sensor applications. In the supervised NPE framework, the output information is utilized to guide the procedures for constructing the adjacent graph and calculating the weight matrix, with which the intrinsic structure of the data can be better described. For performance evaluation of the proposed method, experiments on a numerical example and an industrial debutanizer column process are carried out. The results show the effectiveness of the proposed framework.

Constrained discriminant neighborhood embedding for high dimensional data feature extraction

Constrained discriminant neighborhood embedding for high dimensional data feature extraction.

Locality Pursuit Embedding

Combination of Local Multiple Patterns and Exponential Discriminant Analysis for facial recognition

Discriminant Neighborhood Embedding for Classification.

Constrained Linear Discrimination Analysis and Face Recognition

Face Feature Extraction and Recognition Via Local Binary Pattern and Two-Dimensional Locality Preserving Projection

Two Dimensional Discriminant Neighborhood Preserving Embedding in Face Recognition

Fuzzy local discriminant embedding for image feature extraction

Face Recognition Using Data Driven Local Appearance Features

Neighbourhood Sensitive Preserving Embedding for Pattern Classification.

Data-Dependent Kernel Discriminant Analysis for Feature Extraction and Classification

By Normalizing To Improve Generalized Foley-Sammon Transform In High-Dimensional Spaces-----With Application To Face Recognition

Enhanced Local Gradient Order Features and Discriminant Analysis for Face Recognition

Efficient Feature Extraction For Image Classification

Local Feature Discriminant Projection

Tied Factors Analysis for High-Dimensional Image Feature Extraction and Recognition Application

Linear Laplacian Discrimination for Feature Extraction

Feature Extraction By Locality-Based Linear Discriminant Analysis

Locality-based Linear Discriminant Projection

Supervised Neighborhood Preserving Embedding for Feature Extraction and Its Application for Soft Sensor Modeling