Generalized Optimal Set with Uncorrelated Discriminant Vectors

曾宪贵,黎绍发,左文明

2004-01-01

Abstract:A novel optimal set of discriminant vectors, i.e., the generalized optimal set of uncorrelated discriminant vectors was presented. The calculation formula to get the set was described too. The experiment results on the ORL huaman face databases show this the method has strong ability in feature extraction.

What problem does this paper attempt to address?

A New Algorithm for Generalized Optimal Discriminant Vectors

Wu Xiaojun,Yang Jingyu,Wang Shitong,Guo Yuefei,Cao Qiying

DOI: https://doi.org/10.1007/bf02947310

2002-01-01

Abstract:A study has been conducted on the algorithm of solving generalized optimal set of discriminant vectors in this paper. This paper proposes an analytical algorithm of solving generalized optimal set of discriminant vectors theoretically for the first time. A lot of computation time can be saved because all the generalized optimal sets of discriminant vectors can be obtained simultaneously with the proposed algorithm, while it needs no iterative operations. The proposed algorithm can yield a much higher recognition rate. Furthermore, the proposed algorithm overcomes the shortcomings of conventional human face recognition algorithms which were effective for small sample size problems only. These statements are supported by the numerical simulation experiments on facial database of ORL.
Discriminant Common Vectors in Principal Component Analysis Transformed Space for Face Recognition

唐亮,熊蓉,刘勇

2009-01-01

Abstract:In this paper, we first offer an efficient algorithm to perform DCV in principal component analysis (PCA) transformed space. In this way, we reduce the algorithm's complexity and improve the efficiency whilst preserving the whole discriminant information. Then, the new algorithm further facilitates us to subtly weight the facial components in PCA space according to corresponding eigenvalues, which is potential to enrich the representative information and thus improves DCV's recognition performance. The experiments conducted on ORL, YALE and AR face database demonstrate the effectiveness of our method.
Robust Feature Extracted by the Generalized Optimal Discriminant Vectors for Face Recognition

Guang Dai,Yuntao Qian,Sen Jia

DOI: https://doi.org/10.1109/wcica.2004.1342259

2004-01-01

Abstract:This paper introduces a novel Gabor direct generalized optimal discriminant vectors (GDG-ODV) method for face recognition (FR). This method can apply directly the generalized optimal discriminant vectors (G-ODV) method that views the optimal set of discriminant vectors as a global transform to the high-dimensional augmented Gabor feature vectors (AGFV) derived from the Gabor wavelet representation of face images. This method has three novelties: 1) it is robust to the facial variations; 2) it can efficiently solve the small sample size problem (SSSP), which widely exists in FR tasks; 3) it can consider the separability of optimal discriminant vectors (ODV) from a global viewpoint. The GDG-ODV method is compared, in terms of classification accuracy, to other common methods on the ORL database. The results indicate that this method is overall superior to those various FR methods.
A NEW ALGORITHM FOR COMPUTING OPTIMAL DISCRIMINANT VECTORS IN CONDITION OF HIGH-DIMENSIONAL DATA

Jiang Yuntao,Guo Yuefei

DOI: https://doi.org/10.3969/j.issn.1000-386X.2010.03.086

2010-01-01

Abstract:For applying the Linear Discriminant Analysis(LDA) method to high-dimensional image data,especially for face recognition,we present a novel method to compute the optimal discriminant vectors without dimension reduction pre-processing on high dimensional image data and can directly compute the optimal discriminant vectors.Discriminant vectors calculated through our method are mutually orthonormal,and have higher discriminating performance comparing with those discriminant vectors derived from existing algorithms.Experimental results on ORL and VALID databases verify the effectiveness of our algorithm.
A novel optimal discriminant principle in high dimensional spaces

Yuefei Guo,Lide Wu

DOI: https://doi.org/10.1109/DEVLRN.2002.1011893

2002-01-01

Abstract:A novel optimal discriminant principle and algorithm in high dimensional space are presented in this paper. The new optimal discriminant vectors have the property: in their spanned space, the within-class distance of training samples equals to zero while the between-class distance doesn't equal to zero. We also illustrate how many optimal discriminant vectors satisfying property above can be obtained. We apply this method to the face recognition and the experimental result shows the performance is superior to the existed methods.
Feature Extraction Method Based on the Generalised Fisher Discriminant Criterion and Facial Recognition

Yue-Fei Guo,Ting-Ting Shu,Jing-Yu Yang,Shi-Jin Li

DOI: https://doi.org/10.1007/pl00010986

2001-01-01

Abstract:: In this paper we view the optimal set of discriminant vectors as a global transform, and consider its separability from a global view point. Based on this idea, the concept of a generalised Fisher discriminant criterion and that of a generalised optimal set of discriminant vectors is introduced. After that, a new algorithm is given to calculate the generalised optimal set of discriminant vectors defined in this paper, which is particularly suited to the case of a small number of samples where the scatter matrix is singular. It is then applied to an experiment on human facial recognition, and the results show that the new algorithm is superior to existing methods in terms of correct classification rate.
Combination of Local Multiple Patterns and Exponential Discriminant Analysis for facial recognition

Lifang Zhou,Bin Fang,Weisheng Li,Lidou Wang

2013-01-01

Abstract:Global features-based methods and local features -based methods have been very successful in face recognition system, yet they can be combined together and jointly optimized so as to minimize the error of a nearest-neighbor classifier. We consider both descriptor for face images with Local Multiple Pattern, and discriminant learning techniques with Exponential Discriminant Analysis. A combination framework based on Local Multiple Pattern and Exponential Discriminant Analysis has been proposed in this paper. Firstly, our approach encodes the multi-scale face feature by Local Multiple Pattern, and then they have been extended to strengthen the discriminative ability by Exponential Discriminant Analysis; Secondly, we suggest to use the above feature on different layers independently so that a multiple classifier system can be attained. Using these techniques, we obtain the state-of-the-art performance on two public available databases. © 2013 IFSA.
Discriminant similarity and variance preserving projection for feature extraction

Pu Huang,Caikou Chen,Zhenmin Tang,Zhangjing Yang

DOI: https://doi.org/10.1016/j.neucom.2014.02.047

IF: 6

2014-01-01

Neurocomputing

Abstract:In this paper, a novel supervised dimensionality reduction algorithm called discriminant similarity and variance preserving projection (DSVPP) is presented for feature extraction and recognition. More specifically, we redefine the intrinsic graph and penalty graph to model the intra-class compactness and inter-class separability of data points, where the intrinsic graph characterizes the similarity information of the same-class points and the penalty graph characterizes the variance information of the not-same-class points. Using the two graphs, the within-class scatter and the between-class scatter are computed, and then a concise feature extraction criterion is raised via minimizing the difference between them. Experimental results on the Wine data set, ORL, FERET and AR face databases show the effectiveness of the proposed method.
Improved kernel direct Fisher discriminant analysis and face recognition

Xiao-run LI,Guang-zhou ZHAO,Liao-ying ZHAO

DOI: https://doi.org/10.3785/j.issn.1008-973X.2008.04.007

2008-01-01

Abstract:An improved kernel direct discriminant analysis(IKDDA) was proposed to compute the optimal discriminant vectors for the kernel-based Fisher discriminant analysis in singular cases.Based on the reproducing kernel theory,the kernel within-class and the kernel between-class scatter matrices were defined,then the Fisher discriminant criterion in the high-dimensional feature space was transformed to the kernel Fisher discriminant criterion.According to the isomorphic mapping theory and the singular value decomposition theorem,the maximum of the kernel Fisher discriminant criterion could be skillfully acquired by solving the minimum of its reciprocal in a small space,thus the final solution could be acquired not to take account of the null space and non-null space of the kernel within-class scatter matrices separately.Experimental results on the ORL and the UMIST face image database indicate that the proposed method can achieve lower error rate and quicker speed compared with other methods.
Orthogonal Vs. Uncorrelated Least Squares Discriminant Analysis for Feature Extraction.

Feiping Nie,Shiming Xiang,Yun Liu,Chenping Hou,Changshui Zhang

DOI: https://doi.org/10.1016/j.patrec.2011.11.028

IF: 4.757

2012-01-01

Pattern Recognition Letters

Abstract:In this paper, a new discriminant analysis for feature extraction is derived from the perspective of least squares regression. To obtain great discriminative power between classes, all the data points in each class are expected to be regressed to a single vector, and the basic task is to find a transformation matrix such that the squared regression error is minimized. To this end, two least squares discriminant analysis methods are developed under the orthogonal or the uncorrelated constraint. We show that the orthogonal least squares discriminant analysis is an extension to the null space linear discriminant analysis, and the uncorrelated least squares discriminant analysis is exactly equivalent to the traditional linear discriminant analysis. Comparative experiments show that the orthogonal one is more preferable for real world applications.
A Generalized Form of Fisher Linear Discriminant Function

Zhengdong Cheng,ZHANG Yu-Jin,FAN Xiang

DOI: https://doi.org/10.3969/j.issn.1003-6059.2009.02.002

2009-01-01

Pattern Recognition and Artificial Intelligence

Abstract:A generalized form of Fisher discriminant function is presented.It overcomes the limitations of two common discriminant functions.The presented form uniforms the discriminant functions in two subspaces of the dual subspace discriminant analysis(DSDA).A new orthogonal discriminant vector set is obtained by QR decomposition,and its discriminant property is approximate to that of the Foley-Sammon orthogonal discriminant vector set with smaller computational complexity.The experiments on ORL and JAFFE database show that theory analysis is consistent to the experimental results.
Constrained Linear Discrimination Analysis and Face Recognition

LI Xiao-run,Guangzhou Zhao,ZHAO Liao-ying

2008-01-01

Abstract:A constrained linear discrimination analysis method was proposed for the feature extraction in the pattern recognition of problems with high dimension and small samples. Applying whitening process and Gram-Schimdt orthogonalization and orthogonal subspace projection, an optimal transformation matrix was designed to minimize the ratio of intra-class distance to inter-class distance while imposing the constraint that different class centers after transformation are along specifically directions that are orthogonal each other. For the small sample problem of face recognition, the whitening process was realized by singular value decomposition. The experimental results using the ORL and the UMIST face image database demonstrate that the effectiveness and performance of CLDA is approximate with some Fisher discrimination analysis such as direct Fisher discrimination analysis (DDA) and regularized Fisher discriminant analysis (R-LDA).
Local Feature Discriminant Projection

Mengyang Yu,Ling Shao,Xiantong Zhen,Xiaofei He

DOI: https://doi.org/10.1109/tpami.2015.2497686

IF: 23.6

2016-01-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:In this paper, we propose a novel subspace learning algorithm called Local Feature Discriminant Projection (LFDP) for supervised dimensionality reduction of local features. LFDP is able to efficiently seek a subspace to improve the discriminability of local features for classification. We make three novel contributions. First, the proposed LFDP is a general supervised subspace learning algorithm which provides an efficient way for dimensionality reduction of large-scale local feature descriptors. Second, we introduce the Differential Scatter Discriminant Criterion (DSDC) to the subspace learning of local feature descriptors which avoids the matrix singularity problem. Third, we propose a generalized orthogonalization method to impose on projections, leading to a more compact and less redundant subspace. Extensive experimental validation on three benchmark datasets including UIUC-Sports, Scene-15 and MIT Indoor demonstrates that the proposed LFDP outperforms other dimensionality reduction methods and achieves state-of-the-art performance for image classification.
Optimal discriminative feature and dictionary learning for image set classification

Guoqing Zhang,Junchuan Yang,Yuhui Zheng,Zhiyuan Luo,Jinglin Zhang

DOI: https://doi.org/10.1016/j.ins.2020.08.066

IF: 8.1

2021-02-01

Information Sciences

Abstract:<p>Image set classification has drawn increasing attention and it has been widely applied to many real-life domains. Due to the existence of multiple images in a set, which contain various view appearance changes, image set classification is a rather challenging task. One potential solution is to learn powerful representations from multiple images to decrease the intra-class diversity and enlarge the inter-class separation. In this paper, we propose an optimal discriminative feature and dictionary learning (ODFDL) method, which attempts to learn a feature mapping matrix and a dictionary such that in the mapped feature space the inter-class sparse reconstruction error of data is maximized and the intra-class sparse reconstruction error is minimized. This learning strategy enforces the learned sparse representations from image sets have large inter-class separation and small intra-class scatter. Furthermore, to better exploit the non-linear information of data from different image sets, we also present two non-linear ODFDL methods, termed Kernel-ODFDL and Hierarchy-ODFDL to further improve the classification performance. Experiments on five commonly used image sets exhibit that our approaches are comparable with many state-of-the-arts.</p>

computer science, information systems
By Normalizing To Improve Generalized Foley-Sammon Transform In High-Dimensional Spaces-----With Application To Face Recognition

Guang Dai,Yuntao Qian

DOI: https://doi.org/10.1109/ICSMC.2004.1400650

2004-01-01

Abstract:Linear discriminant analysis (LDA) is an effective feature extraction technique for classification. A new LDA-based algorithm, i.e., direct normalized generalized Foley-Sammon transform (DN-GFST) method in high dimensional spaces, is presented in this paper. It not only overcomes the limitation of traditional LDA that they overemphasize the larger distance between classes and cause large overlaps of neighboring classes, but also has the best separable ability in global sense. Lastly, our method is applied to facial image recognition, and the experimental results show that the performance of the present method is superior to those of the existing methods in terms of the classification error rate.
Data-Dependent Kernel Discriminant Analysis for Feature Extraction and Classification

Jun-Bao Li,Jeng-Shyang Pan,Zhe-Ming Lu,Bin-Yih Liao

DOI: https://doi.org/10.1109/icia.2006.305931

2006-01-01

Abstract:Subspace analysis is an effective technique for feature extraction, which aims at finding a low-dimensional space of high-dimensional data. In this paper, a novel subspace analysis method based on Data-Dependent Kernel Discriminant Analysis (DDKDA) is proposed for dimension reduction. The procedure of DDKDA contains two stages: one is to find the optimal combination coefficients by solving a constrained optimization function which transformed to an eigenvalue problem; other is to implement KDA under the optimal data-dependent kernel with Fisher criterion. DDKDA is more adaptive to the input data than KDA owing to the optimization of projection from input space to feature space with the data-dependent kernel, which enhances the performance of KDA. Experiments on the ORL and Yale face databases demonstrate the good performance of the proposed algorithm.
An Improved Generalized Discriminant Analysis for Large-Scale Data Set

Weiya Shi,Yue-Fei Guo,Cheng Jin,Xiangyang Xue

DOI: https://doi.org/10.1109/ICMLA.2008.41

2008-01-01

Abstract:In order to overcome the computation and storage problem for large-scale data set, an efficient iterative method of Generalized Discriminant Analysis is proposed. Because sample vectors cannot explicitly be denoted in kernel space, some mathematical tricks are firstly used to transform the kernel matrix. Then, the columns of transformed matrix are used for iterative algorithm to extract nonlinear discriminant vectors. The proposed method reduces space complexity from o(m^2) to o(m) and its effectiveness is validated from experimental results.
Generalized robust linear discriminant analysis for jointly sparse learning

Yufei Zhu,Zhihui Lai,Can Gao,Heng Kong

DOI: https://doi.org/10.1007/s10489-024-05632-6

IF: 5.3

2024-07-19

Applied Intelligence

Abstract:Linear discriminant analysis (LDA) is a well-known supervised method that can perform dimensionality reduction and feature extraction effectively. However, traditional LDA-based methods need to be turned into the trace ratio form to compute the closed-form solution, in which the within-class scatter matrix should be nonsingular. In this article, we design a new model named generalized robust linear discriminant analysis (GRLDA) method to tackle this disadvantage and improve the robustness. GRLDA uses -norm on both loss functions to reduce the influence of outliers and on regularization term to obtain joint sparsity simultaneously. The intrinsic graph and the penalty graph are constructed to characterize the intraclass similarity and interclass separability, respectively. A novel optimization method is proposed to solve the proposed model, in which a quadratic problem on the Stiefel manifold is involved to avoid the inverse computation on a singular matrix. We also analyze the computational complexity rigorously. Finally, the experimental results on face, object, and medical images exhibit the superiority of GRLDA.

computer science, artificial intelligence
GO-LDA: Generalised Optimal Linear Discriminant Analysis

Jiahui Liu,Xiaohao Cai,Mahesan Niranjan

2023-05-24

Abstract:Linear discriminant analysis (LDA) has been a useful tool in pattern recognition and data analysis research and practice. While linearity of class boundaries cannot always be expected, nonlinear projections through pre-trained deep neural networks have served to map complex data onto feature spaces in which linear discrimination has served well. The solution to binary LDA is obtained by eigenvalue analysis of within-class and between-class scatter matrices. It is well known that the multiclass LDA is solved by an extension to the binary LDA, a generalised eigenvalue problem, from which the largest subspace that can be extracted is of dimension one lower than the number of classes in the given problem. In this paper, we show that, apart from the first of the discriminant directions, the generalised eigenanalysis solution to multiclass LDA does neither yield orthogonal discriminant directions nor maximise discrimination of projected data along them. Surprisingly, to the best of our knowledge, this has not been noted in decades of literature on LDA. To overcome this drawback, we present a derivation with a strict theoretical support for sequentially obtaining discriminant directions that are orthogonal to previously computed ones and maximise in each step the Fisher criterion. We show distributions of projections along these axes and demonstrate that discrimination of data projected onto these discriminant directions has optimal separation, which is much higher than those from the generalised eigenvectors of the multiclass LDA. Using a wide range of benchmark tasks, we present a comprehensive empirical demonstration that on a number of pattern recognition and classification problems, the optimal discriminant subspaces obtained by the proposed method, referred to as GO-LDA (Generalised Optimal LDA), can offer superior accuracy.

Computer Vision and Pattern Recognition,Numerical Analysis
Optimal projections for Gaussian discriminants

David P. Hofmeyr,Francois Kamper,Michail C. Melonas

DOI: https://doi.org/10.1007/s11634-021-00486-z

2022-01-04

Advances in Data Analysis and Classification

Abstract:We study the problem of obtaining optimal projections for performing discriminant analysis with Gaussian class densities. Unlike in most existing approaches to the problem, we focus on the optimisation of the multinomial likelihood based on posterior probability estimates, which directly captures discriminability of classes. Finding optimal projections offers utility for dimension reduction and regularisation, as well as instructive visualisation for better model interpretability. Practical applications of the proposed approach show that it is highly competitive with existing Gaussian discriminant models. Code to implement the proposed method is available in the form of an R package from https://github.com/DavidHofmeyr/OPGD.

statistics & probability

Generalized Optimal Set with Uncorrelated Discriminant Vectors

A New Algorithm for Generalized Optimal Discriminant Vectors

Discriminant Common Vectors in Principal Component Analysis Transformed Space for Face Recognition

Robust Feature Extracted by the Generalized Optimal Discriminant Vectors for Face Recognition

A NEW ALGORITHM FOR COMPUTING OPTIMAL DISCRIMINANT VECTORS IN CONDITION OF HIGH-DIMENSIONAL DATA

A novel optimal discriminant principle in high dimensional spaces

Feature Extraction Method Based on the Generalised Fisher Discriminant Criterion and Facial Recognition

Combination of Local Multiple Patterns and Exponential Discriminant Analysis for facial recognition

Discriminant similarity and variance preserving projection for feature extraction

Improved kernel direct Fisher discriminant analysis and face recognition

Orthogonal Vs. Uncorrelated Least Squares Discriminant Analysis for Feature Extraction.

A Generalized Form of Fisher Linear Discriminant Function

Constrained Linear Discrimination Analysis and Face Recognition

Local Feature Discriminant Projection

Optimal discriminative feature and dictionary learning for image set classification

By Normalizing To Improve Generalized Foley-Sammon Transform In High-Dimensional Spaces-----With Application To Face Recognition

Data-Dependent Kernel Discriminant Analysis for Feature Extraction and Classification

An Improved Generalized Discriminant Analysis for Large-Scale Data Set

Generalized robust linear discriminant analysis for jointly sparse learning

GO-LDA: Generalised Optimal Linear Discriminant Analysis

Optimal projections for Gaussian discriminants