Abstract:The recent literature indicates that preserving global pairwise sample similarity is of great importance for feature selection and that many existing selection criteria essentially work in this way. In this paper, we argue that besides global pairwise sample similarity, the local geometric structure of data is also critical and that these two factors play different roles in different learning scenarios. In order to show this, we propose a global and local structure preservation framework for feature selection (GLSPFS) which integrates both global pairwise sample similarity and local geometric data structure to conduct feature selection. To demonstrate the generality of our framework, we employ methods that are well known in the literature to model the local geometric data structure and develop three specific GLSPFS-based feature selection algorithms. Also, we develop an efficient optimization algorithm with proven global convergence to solve the resulting feature selection problem. A comprehensive experimental study is then conducted in order to compare our feature selection algorithms with many state-of-the-art ones in supervised, unsupervised, and semisupervised learning scenarios. The result indicates that: 1) our framework consistently achieves statistically significant improvement in selection performance when compared with the currently used algorithms; 2) in supervised and semisupervised learning scenarios, preserving global pairwise similarity is more important than preserving local geometric data structure; 3) in the unsupervised scenario, preserving local geometric data structure becomes clearly more important; and 4) the best feature selection performance is always obtained when the two factors are appropriately integrated. In summary, this paper not only validates the advantages of the proposed GLSPFS framework but also gains more insight into the information to be preserved in different feature selection tasks.

Preserving Similarity and Staring Decisis for Feature Selection

$$\Hbox {u}^2\hbox {f}^2\hbox {S}^2$$ U 2 F 2 S 2 : Uncovering Feature-level Similarities for Unsupervised Feature Selection.

U^2F^2S^2 : Uncovering Feature-level Similarities for Unsupervised Feature Selection

Feature Selection with Conditional Mutual Information Considering Feature Interaction

A Method of Discriminative Information Preservation and In-Dimension Distance Minimization Method for Feature Selection

Feature Selection Based on Quality of Information.

R2CI: Information theoretic-guided feature selection with multiple correlations

A Survey for Study of Feature Selection Algorithms

Feature Selection Considering Two Types of Feature Relevancy and Feature Interdependency.

Using cooperative game theory to optimize the feature selection problem

Class-specific Mutual Information Variation for Feature Selection.

Feature Selection Using Mutual Information: An Experimental Study

Multi-class Feature Selection by Exploring Reliable Class Correlation

An Information-Theoretic Approach to Universal Feature Selection in High-Dimensional Inference.

Global and Local Structure Preservation for Feature Selection

Supervised Feature Selection by Preserving Class Correlation

An Incremental Feature Selection Approach Based on Information Entropy for Incomplete Data

Feature Selection Considering Weighted Relevancy

Information-theoretic Feature Selection Based on the Weight of the New Classification Information

Feature Selection: A Data Perspective

Feature evaluation and selection with cooperative game theory