Abstract:Unsupervised feature selection is fundamental in statistical pattern recognition, and has drawn persistent attention in the past several decades. Recently, much work has shown that feature selection can be formulated as nonlinear dimensionality reduction with discrete constraints. This line of research emphasizes utilizing the manifold learning techniques, where feature selection and learning can be studied based on the manifold assumption in data distribution. Many existing feature selection methods such as Laplacian score, SPEC (spectrum decomposition of graph Laplacian), TR (trace ratio) criterion, MSFS (multi-cluster feature selection) and EVSC (eigenvalue sensitive criterion) apply the basic properties of graph Laplacian, and select the optimal feature subsets which best preserve the manifold structure defined on the graph Laplacian. In this paper, we propose a new feature selection perspective from locally linear embedding (LLE), which is another popular manifold learning method. The main difficulty of using LLE for feature selection is that its optimization involves quadratic programming and eigenvalue decomposition, both of which are continuous procedures and different from discrete feature selection. We prove that the LLE objective can be decomposed with respect to data dimensionalities in the subset selection problem, which also facilitates constructing better coordinates from data using the principal component analysis (PCA) technique. Based on these results, we propose a novel unsupervised feature selection algorithm, called locally linear selection (LLS), to select a feature subset representing the underlying data manifold. The local relationship among samples is computed from the LLE formulation, which is then used to estimate the contribution of each individual feature to the underlying manifold structure. These contributions, represented as LLS scores, are ranked and selected as the candidate solution to feature selection. We further develop a locally linear rotation-selection (LLRS) algorithm which extends LLS to identify the optimal coordinate subset from a new space. Experimental results on real-world datasets show that our method can be more effective than Laplacian eigenmap based feature selection methods.

Patterns classification of nonlinear multi-dimensional time series based on manifold learning

Learning Exemplar-Represented Manifolds in Latent Space for Classification.

Rotating Machinery Fault Diagnosis Based on Manifold Learning Using Semi-supervised Local Linear Embedding

Sensor Fault Diagnosis of Nonlinear Processes Based on Structured Kernel Principal Component Analysis

Neighbourhood Sensitive Preserving Embedding for Pattern Classification.

Manifold Analysis In Reconstructed State Space For Nonlinear Signal Classification

Bearing Fault Diagnosis Based on Statistical Locally Linear Embedding

Time Neighborhood Preserving Embedding Model And Its Application For Fault Detection

Nonlinear Dynamic System Based on SAE-LDS Model for Fault Diagnosis

An integrated manifold learning approach for high-dimensional data feature extractions and its applications to online process monitoring of additive manufacturing

Generalized Orthogonal Locality Preserving Projections for Nonlinear Fault Detection and Diagnosis

A novel fault diagnosis system using pattern classification on kernel FDA subspace

Nonlinear Fault Detection Based on Locally Linear Embedding

Unsupervised Image Classifier based on Manifold Learning

Detecting Local Manifold Structure for Unsupervised Feature Selection

Supervised Manifold Learning Based on Multi-Feature Information Discriminative Fusion within an Adaptive Nearest Neighbor Strategy Applied to Rolling Bearing Fault Diagnosis

Deep Clustering Bearing Fault Diagnosis Method Based on Local Manifold Learning of an Autoencoded Embedding

Semi-Supervised Machinery Health Assessment Framework via Temporal Broad Learning System Embedding Manifold Regularization with Unlabeled Data

Unified Discriminant Manifold Learning for Rotating Machinery Fault Diagnosis

Improved Multi-Distance ARMF Integrated With LTSA Based Pattern Matching Method and Its Application in Fault Diagnosis

Structured Manifold Broad Learning System: A Manifold Perspective for Large-Scale Chaotic Time Series Analysis and Prediction