A Fast Evolutionary Pursuit Algorithm Based on Linearly Combining Vectors.

QJ Zhao,HT Lu,D Zhang

DOI: https://doi.org/10.1016/j.patcog.2005.09.001

IF: 8

2006-01-01

Pattern Recognition

Abstract:In this paper, we propose a fast Evolutionary pursuit algorithm. It searches for optimal discriminant vectors through linearly combining standard bases. Experiments on the Yale and ORL databases verify its effectiveness and enhanced efficiency.

What problem does this paper attempt to address?

A direct evolutionary feature extraction algorithm for classifying high dimensional data

Qijun Zhao,David Zhang,Hongtao Lu

2006-01-01

Abstract:Among various feature extraction algorithms, those based on genetic algorithms are promising owing to their potential parallelizability and possible applications in large scale and high dimensional data classification. However, existing genetic algorithm based feature extraction algorithms are either limited in searching optimal projection basis vectors or costly in both time and space complexities and thus not directly applicable to high dimensional data. In this paper, a direct evolutionary feature extraction algorithm is proposed for classifying high-dimensional data. It constructs projection basis vectors using the linear combination of the basis of the search space and the technique of orthogonal complement. It also constrains the search space when seeking for the optimal projection basis vectors. It evaluates individuals according to the classification performance on a subset of the training samples and the generalization ability Df the projection basis vectors represented by the individuals. We compared the proposed algorithm with some representative feature extraction algorithms in face recognition, including the evolutionary pursuit algorithm, Eigenfaces, and Fisherfaces. The results on the widely-used Yale and ORL face databases show that the proposed algorithm has an excellent performance in classification while reducing the space complexity by an order of magnitude.
Combination of Local Multiple Patterns and Exponential Discriminant Analysis for facial recognition

Lifang Zhou,Bin Fang,Weisheng Li,Lidou Wang

2013-01-01

Abstract:Global features-based methods and local features -based methods have been very successful in face recognition system, yet they can be combined together and jointly optimized so as to minimize the error of a nearest-neighbor classifier. We consider both descriptor for face images with Local Multiple Pattern, and discriminant learning techniques with Exponential Discriminant Analysis. A combination framework based on Local Multiple Pattern and Exponential Discriminant Analysis has been proposed in this paper. Firstly, our approach encodes the multi-scale face feature by Local Multiple Pattern, and then they have been extended to strengthen the discriminative ability by Exponential Discriminant Analysis; Secondly, we suggest to use the above feature on different layers independently so that a multiple classifier system can be attained. Using these techniques, we obtain the state-of-the-art performance on two public available databases. © 2013 IFSA.
Evolutionary Discriminant Feature Extraction with Application to Face Recognition

Qijun Zhao,David Zhang,Lei Zhang,Hongtao Lu

DOI: https://doi.org/10.1155/2009/465193

2009-01-01

EURASIP Journal on Advances in Signal Processing

Abstract:Evolutionary computation algorithms have recently been explored to extract features and applied to face recognition. However these methods have high space complexity and thus are not efficient or even impossible to be directly applied to real world applications such as face recognition where the data have very high dimensionality or very large scale. In this paper, we propose a new evolutionary approach to extracting discriminant features with low space complexity and high search efficiency. The proposed approach is further improved by using the bagging technique. Compared with the conventional subspace analysis methods such as PCA and LDA, the proposed methods can automatically select the dimensionality of feature space from the classification viewpoint. We have evaluated the proposed methods in comparison with some state-of-the-art methods using the ORL and AR face databases. The experimental results demonstrated that the proposed approach can successfully reduce the space complexity and enhance the recognition performance. In addition, the proposed approach provides an effective way to investigate the discriminative power of different feature subspaces.
A Fast Vector Quantization Encoding Algorithm Based on Adaptive Searching Range and Sequence

LU Zhe-ming,PAN Jeng-shyang,SUN Sheng-he

DOI: https://doi.org/10.3321/j.issn:0372-2112.2000.08.026

2000-01-01

Tien Tzu Hsueh Pao/Acta Electronica Sinica

Abstract:A novel fast encoding algorithm was presented for vector quantization, in which, four characteristic values are computed for each codeword before encoding, then four ordered codebooks are obtained according to the ascending order of four characteristic values respectively. During the encoding process the adaptive searching range and sequence are obtained for each input vector in order to reject a lot of unmatched codewords. The test result shows that this algorithm enables us to search merely 3% to 8% of the codewords to encode an input vector while preserving its coding performance very close to that of the full search method; the encoding time is reduced by about 93% as compared with full search method.
High-dimensional Feature Selection in Classification: A Length-Adaptive Evolutionary Approach.

Junhai Zhou,Jianyun Lu,Quanwang Wu,Junhao Wen

DOI: https://doi.org/10.1109/icnsc55942.2022.10004048

2022-01-01

Abstract:Feature selection is an essential technique which has been widely applied in data mining. Recent research has shown that a good feature subset can be obtained by using evolutionary computing (EC) approaches as a wrapper. However, most feature selection methods based on EC use a fixed-length encoding to represent feature subsets. When this fixed length representation is applied to high-dimensional data, it requires a large amount of memory space as well as a high computational cost. Moreover, this representation is inflexible and may limit the performance of EC because of a too huge search space. In this paper, we propose an Adaptive- Variable-Length Genetic Algorithm (A VLGA), which adopts a variable-length individual encoding and enables individuals with different lengths in a population to evolve in their own search space. An adaptive length changing mechanism is introduced which can extend or shorten an individual to guide it to explore in a better search space. Thus, A VLGA is able to adaptively concentrate on a smaller but more fruitful search space and yield better solutions more quickly. Experimental results on 6 high-dimensional datasets reveal that A VLGA performs significantly better than existing methods.
Fast codevector search scheme for 3D mesh model vector quantisation

Li, Z.,Lu, Z.-M.

DOI: https://doi.org/10.1049/el:20082270

2008-01-01

Electronics Letters

Abstract:A new fast approach to the nearest codevector search for 3D mesh compression using an orthonormal transformed codebook is proposed. The algorithm uses the coefficients of an input vector along a set of orthonormal bases as the criteria to reject impossible codevectors. Compared to the full search algorithm, a great deal of computational time is saved without extra distortion and additional storage requirement. Simulation results demonstrate the effectiveness of the proposed algorithm in terms of computational complexity.
Equal-Average Equal-Variance Nearest Neighbor Search Algorithm Based on Hadamard Transform

ZM Lu,H Pei

DOI: https://doi.org/10.1109/icmlc.2003.1260084

2004-01-01

Abstract:A new fast nearest neighbor codeword search algorithm for image vector quantization (VQ) is introduced. This algorithm uses two significant features of a hadamard transformed vector, that is, the average, the variance, to eliminate more unmatched code words. It saves a great deal of computational time and distortion calculations. Experimental results demonstrate the performance of the proposed algorithm is good.
A Fast Evolutionary Algorithm for Combinatorial Optimization Problems

XS Yan,H Li,ZH Cai,LS Kang

DOI: https://doi.org/10.1109/icmlc.2005.1527510

2005-01-01

Abstract:In this paper we proposed a new algorithm based on inver-over operator, for combinatorial optimization problems. Inver-over is based on simple inversion; however, knowledge taken from other individuals in the population influences its action. In the new algorithm we use some new strategies including selection operator, replace operator and some new control strategy, which have been proved to be very efficient to accelerate the converge speed. Through the experiment, the new algorithm shows great efficiency in solving TSP with the problem scale under 300.
Bagging Evolutionary Feature Extraction Algorithm For Classification

Tianwen Zhao,Qijun Zhao,Hongtao Lu,David Zhang

DOI: https://doi.org/10.1109/ICNC.2007.280

2007-01-01

Abstract:Feature extraction is significant for pattern analysis and classification. Those based on genetic algorithms are promising owing to their potential parallelizability and possible applications in large scale and high dimensional data classification. Most recently, Zhao et al. presented a direct evolutionary feature extraction algorithm(DEFE) which can reduce the space complexity and improve the efficiency, thus overcoming the limitations of many genetic algorithm based feature extraction algorithms(EFE). However, DEFE does not consider the outlier problem which could deteriorate the classification performance, especially when the training sample set is small. Moreover when there are many classes, the null space of within-class scatter matrix(S-omega) becomes small, resulting in poor discrimination performance in that space. In this paper we propose a bagging evolutionary feature extraction algorithm(BEFE) incorporating bagging into a revised DEFE algorithm to improve the DEFE's performance in cases of small training sets and large number of classes. The proposed algorithm has been applied to face recognition and testified using the Yale and ORL face databases. Feature extraction is significant for pattern analysis and classification. Those based on genetic algorithms are promising owing to their potential parallelizability and possible applications in large scale and high dimensional data classification. Most recently, Zhao et al. presented a direct evolutionary feature extraction algorithm(DEFE) which can reduce the space complexity and improve the efficiency, thus overcoming the limitations of many geneti. c algorithm basedJeature extraction algorithms(EFE). However, DEFE does not consider the outlier problem which could deteriorate the classification performance, especially when the training sample set is small. Moreover when there are many classes, the null space of within-class scatter matrix(S-omega) becomes small, resulting in poor discrimination performance in that space. In this paper we propose a bagging evolutionary feature extraction algorithm(BEFE) incorporating bagging into a revised DEFE algorithm to improve the DEFE's performance in cases of small training sets and large number of classes. The proposed algorithm has been applied to face recognition and testified using the Yale and ORLJace databases.
Evolutionary Strategy of Lexicographic Order for Combinational Problem

肖赤心,蔡自兴,王勇

2010-01-01

Abstract:In order to construct a fast and effective algorithm to solve large-scale combinational problems in desirable computational time rather than be trapped in weakness as some existing algorithms,a novel encoding approach is proposed in this paper which applies an one to one mapping from a discrete space to a continuous integer section.Assembled with successful exploration and exploitation mechanism of evolutionary strategy,the performances of the algorithm are largely promoted.Since the one to one mapping between codes and combinational vectors,the new scheme only provides feasible solutions,which can help to avoid redundant computation existing in some algorithms effectively and the search space is further reduced.Secondly,a queue of elites is added in evolutionary mechanism combined with some particular learning strategy.The queue is refreshed frequently in evolution.This can help the algorithm to maintain better gene blocks.Finally,its convergence to global optimal solution with probability one is proved.The numerical experiments based on the Benchmarks of traveling salesman problem library(TSPLIB)show the effectiveness of algorithm proposed.
An Improvement Of Fast Search Algorithm For Vector Quantization

Mc Chung,Sc Chen,Ct Yu,Py Chen

DOI: https://doi.org/10.1109/ispacs.2005.1595355

2005-01-01

Abstract:One of the major issues of vector quantization encoding is to reduce the computational complexity from searching the best-matched codevector, so as to shorten the search time. In this paper, we extended the partial-sum algorithm to exclude much more codevectors from doing the complex Euclidean distance computation. An optimal trade-off has been found between the computational complexity and the search space. According to the experimental results, our algorithms can significantly reduce the number of redundant codevectors, which do not need to do the distance computation, while obtaining the same encoding quality as that of full search algorithm.
A convergent solution to two dimensional linear discriminant analysis

Wei Chen,Kaiqi Huang,Tieniu Tan,Dacheng Tao

DOI: https://doi.org/10.1109/ICIP.2009.5413462

2009-01-01

Abstract:The matrix based data representation has been recognized to be effective for face recognition because it can deal with the undersampled problem. One of the most popular algorithms, the two dimensional linear discriminant analysis (2DLDA), has been identified to be effective to encode the discriminative information for training matrix represented samples. However, 2DLDA does not converge in the training stage. This paper presents an evolutionary computation based solution, referred to as E-2DLDA, to provide a convergent training stage for 2DLDA. In E-2DLDA, every randomly generated candidate projection matrices are first normalized. The evolutionary computation method optimizes the projection matrices to best separate different classes. Experimental results show E-2DLDA is convergent and outperforms 2DLDA.
A binary individual search strategy-based bi-objective evolutionary algorithm for high-dimensional feature selection

Tao Li,Zhi-Hui Zhan,Jiu-Cheng Xu,Qiang Yang,Yuan-Yuan Ma

DOI: https://doi.org/10.1016/j.ins.2022.07.183

IF: 8.1

2022-09-01

Information Sciences

Abstract:Evolutionary computation is promising in tackling with the feature selection problem, but still has poor performance in obtaining good feature subset in high-dimensional problems. In order to efficiently obtain the optimal feature subset with higher classification accuracy and lower feature dimensions, a binary individual search strategy-based bi-objective evolutionary algorithm is proposed. The proposed algorithm has three advantages and contributions. Firstly, an improved fisher score is utilized to preprocess the feature space to remove the irrelevant and redundant features. It can decrease the feature dimensionality and compress the search space of feature subset effectively. Secondly, a binary individual search strategy is developed that contains a nearest neighbor binary individual crossover operator and an adaptive binary individual mutation operator, which can search the global optimal feature combination. Thirdly, enhanced population entropy and improved average convergence rate are adopted to monitor the correlation between the diversity of the population and the convergence of optimization objectives. Promising experimental results on twelve high-dimensional datasets reveal that the proposed algorithm can obtain competitive classification accuracy and effectively reduce the size of feature subset compared with ten state-of-the-art evolutionary algorithms.

computer science, information systems
Enhancing the Precision of Redundant Data in Feature Extraction by an Improved Evolutionary Algorithm

Xin Qian,Shujuan Wang,Guotao Wang

DOI: https://doi.org/10.1109/icbda.2017.8078770

2017-01-01

Abstract:Evolutionary algorithms' superior exploration skills should make them promising candidates for handling optimization problems involving big data. In order to enhance the local exploration of the traditional evolutionary algorithms, normally either selection or mutation operation is chosen. Here, an improved crossover operation (ICO) as a new evolutionary algorithm is offered. According to the biological phenomenon of sexual selection, the excellent parents are chosen for the crossover operation in higher probability. The local exploration of the algorithm is sharply enhanced. The convergence precision of the algorithm is improved by an order of magnitude for the common eight test formula examples. A non-periodical function containing redundant data with random noise is chosen. The nuclear least squares method is used for data extraction. The new algorithm effectively prevent over fitting, improving the robustness of the algorithm.
Parsimonious feature extraction based on genetic algorithms and support vector machines

Qijun Zhao,Hongtao Lu,David Zhang

DOI: https://doi.org/10.1007/11759966_206

2006-01-01

Abstract:Most existing feature extraction algorithms aim at best preserving information in the original data or at improving the separability of data, but fail to consider the possibility of further reducing the number of used features. In this paper, we propose a parsimonious feature extraction algorithm. Its motivation is using as few features as possible to achieve the same or even better classification performance. It searches for the optimal features using a genetic algorithm and evaluates the features referring to Support Vector Machines. We tested the proposed algorithm by face recognition on the Yale and FERET databases. The experimental results proved its effectiveness and demonstrated that parsimoniousness should be a significant factor in developing efficient feature extraction algorithms.
Base vector selection for kernel matching pursuit

Qing Li,Licheng Jiao

DOI: https://doi.org/10.1007/11811305_105

2006-01-01

Abstract:Kernel matching pursuit (KMP) is a relatively new learning algorithm to produce non-linear version of conventional supervised and unsupervised learning algorithm. However, it also contains some defects such as storage problem (in training process) and sparsity problem. In this paper, a new method is proposed to pre-select the base vectors from the original data according to vector correlation principle, which could greatly reduce the scale of the optimization problem and improve the sparsity of the solution. The method could capture the structure of the data space by approximating a basis of the subspace of the data; therefore, the statistical information of the training samples is preserved. In the paper, the deduction of mathematical process is given in details and the number of simulation results on artificial data and practical data has been done to validate the performance of base vector selection (BVS) algorithm. The experimental results show the combination of such algorithm with KMP can make great progress while leave the performance almost unchanged.
Linear discriminant analysis based on genetic algorithm

Daoyi Shen,Yanwei Pang,Lei Wang,Nenghai Yu

2008-01-01

Abstract:Since the objective function of Linear discriminant analysis (LDA) is not the training classification error on the Fisher criterion. In some cases that the Fisher criterion does not represent the minimum training error, LDA fails to find the optimal feature subspace for classification. To solve the problem, by analyzing the relation between the data distribution and the projection directions of LDA, the relation between the project-vector of LDA and the eigenvalues of the inter-class scatter matrix and the intra-class scatter matrix is revealed. Based on this observation, a modified LDA method is presented based on the genetic algorithm. Aimed at the minimum training classification error, by using the genetic algorithm, the method pertinently adjusts the eigenvalues of the inter-class scatter matrix to find the optimal feature subspace. Experiments on both synthetic data and real data show that the method is superior to other linear subspace methods.
A Simple but Efficient Way to Combine Vlad with Locality-Constrained Linear Coding

Zhenglin Tan,Wenmin Wang,Yifeng Jiang,Ronggang Wang

DOI: https://doi.org/10.1109/vcip.2016.7805518

2016-01-01

Abstract:The VLAD (vector of locally aggregated descriptors) representation, derived from BoF and Fisher kernel, has shown its efficiency in the field of image search. However, assigning local descriptors to a codeword is a hard voting process, which does not consider the uncertainty and the plausibility for single codeword. In this paper, we propose an approach to combine VLAD with locality-constrained linear coding, as opposed to the original one, considering several nearest neighbors when assigning local descriptors and computing weights. In order to evaluate our proposed method, experiments are conducted on several image classification benchmarks, using VLAD for comparison. The experimental results show that our method stably outperforms VLAD in terms of classification accuracy, while producing feature representation of the same dimension without much additional computational cost.
GleanVec: Accelerating vector search with minimalist nonlinear dimensionality reduction

Mariano Tepper,Ishwar Singh Bhati,Cecilia Aguerrebere,Ted Willke

2024-10-15

Abstract:Embedding models can generate high-dimensional vectors whose similarity reflects semantic affinities. Thus, accurately and timely retrieving those vectors in a large collection that are similar to a given query has become a critical component of a wide range of applications. In particular, cross-modal retrieval (e.g., where a text query is used to find images) is gaining momentum rapidly. Here, it is challenging to achieve high accuracy as the queries often have different statistical distributions than the database vectors. Moreover, the high vector dimensionality puts these search systems under compute and memory pressure, leading to subpar performance. In this work, we present new linear and nonlinear methods for dimensionality reduction to accelerate high-dimensional vector search while maintaining accuracy in settings with in-distribution (ID) and out-of-distribution (OOD) queries. The linear LeanVec-Sphering outperforms other linear methods, trains faster, comes with no hyperparameters, and allows to set the target dimensionality more flexibly. The nonlinear Generalized LeanVec (GleanVec) uses a piecewise linear scheme to further improve the search accuracy while remaining computationally nimble. Initial experimental results show that LeanVec-Sphering and GleanVec push the state of the art for vector search.

Information Retrieval,Machine Learning
A Compact Binary Aggregated Descriptor Via Dual Selection for Visual Search

Yuwei Wu,Zhe Wang,Junsong Yuan,Ling-Yu Duan

DOI: https://doi.org/10.1145/2964284.2967256

2016-01-01

Abstract:To achieve high retrieval accuracy over a large scale image/video dataset, recent research efforts have demonstrated that employing extremely high-dimensional descriptors such as the Fisher Vector (FV) and the Vector of Locally Aggregated Descriptors (VLAD) can yield good performance. To enable fast search, the FV (or VLAD) is usually compressed by product quantization (PQ) or hashing. However, compressing high-dimensional descriptors via PQ or hashing may become intractable and infeasible due to both the storage and computation requirements for the linear/nonlinear projection of PQ or hashing methods. We develop a novel compact aggregated descriptor via dual selection for visual search. We utilize both sample-specific Gaussian component redundancy and bit dependency within a binary aggregated descriptor to produce its compact binary codes. The proposed method can effectively reduce the codesize of the raw aggregated descriptors, without degrading the search accuracy or introducing additional memory footprint. We demonstrate the significant advantages of the proposed binary codes in solving the approximate nearest neighbor (ANN) visual search problem. Experimental results on extensive datasets show that our method outperforms the state-of-the-art methods.

A Fast Evolutionary Pursuit Algorithm Based on Linearly Combining Vectors.

A direct evolutionary feature extraction algorithm for classifying high dimensional data

Combination of Local Multiple Patterns and Exponential Discriminant Analysis for facial recognition

Evolutionary Discriminant Feature Extraction with Application to Face Recognition

A Fast Vector Quantization Encoding Algorithm Based on Adaptive Searching Range and Sequence

High-dimensional Feature Selection in Classification: A Length-Adaptive Evolutionary Approach.

Fast codevector search scheme for 3D mesh model vector quantisation

Equal-Average Equal-Variance Nearest Neighbor Search Algorithm Based on Hadamard Transform

A Fast Evolutionary Algorithm for Combinatorial Optimization Problems

Bagging Evolutionary Feature Extraction Algorithm For Classification

Evolutionary Strategy of Lexicographic Order for Combinational Problem

An Improvement Of Fast Search Algorithm For Vector Quantization

A convergent solution to two dimensional linear discriminant analysis

A binary individual search strategy-based bi-objective evolutionary algorithm for high-dimensional feature selection

Enhancing the Precision of Redundant Data in Feature Extraction by an Improved Evolutionary Algorithm

Parsimonious feature extraction based on genetic algorithms and support vector machines

Base vector selection for kernel matching pursuit

Linear discriminant analysis based on genetic algorithm

A Simple but Efficient Way to Combine Vlad with Locality-Constrained Linear Coding

GleanVec: Accelerating vector search with minimalist nonlinear dimensionality reduction

A Compact Binary Aggregated Descriptor Via Dual Selection for Visual Search