Abstract:High-dimensional data have been a challenging problem in classification. Feature selection works as a filter to remove irrelevant or redundant features and has made comparative progress. However, this problem is still challenging because current methods consider only the correlation between two variables while leaving the correlation among multiple variables largely unsolved, and multivariate interactions can contain joint information that cannot be obtained pairwise. Furthermore, many feature selection methods require hyperparameter settings, which require prior knowledge and lack interpretability. Focusing on the above problems, this paper proposes the total correlation information coefficient-based feature selection (TCIC_FS) method to select the optimal solution, which can avoid setting hyperparameters and fully consider the correlations among multiple variables. First, based on a Gaussian copula, the total correlation information coefficient (TCIC) is proposed to evaluate the correlations among multiple variables. Compared with the existing multivariate correlation methods, TCIC can measure a wider range of multivariate correlations, including linear, nonlinear, functional, and nonfunctional correlations. Second, a novel evaluation mechanism based on TCIC is proposed to measure the relevance between features and classes and the redundancy between a single feature and a selected feature subset. Finally, the TCIC_FS method is constructed based on the TCIC and the evaluation mechanism. Compared with the baseline values, the TCIC_FS method has the lowest time complexity and the smallest optimal feature subset obtained by single selection. Therefore, TCIC_FS is more suitable for processing high-dimensional data.

An Information-Theoretic Approach to Unsupervised Feature Selection for High-Dimensional Data

$$\Hbox {u}^2\hbox {f}^2\hbox {S}^2$$ U 2 F 2 S 2 : Uncovering Feature-level Similarities for Unsupervised Feature Selection.

Unsupervised Feature Selection with Ordinal Locality.

U^2F^2S^2 : Uncovering Feature-level Similarities for Unsupervised Feature Selection

An Information-Theoretic Approach to Universal Feature Selection in High-Dimensional Inference.

Unsupervised feature selection with high-order similarity learning

Low-rank Unsupervised Graph Feature Selection Via Feature Self-Representation.

Maximum Correntropy Criterion-Based Sparse Subspace Learning for Unsupervised Feature Selection

Nonlinear Joint Unsupervised Feature Selection

Subspace Learning for Unsupervised Feature Selection Via Matrix Factorization.

Understanding-Oriented Unsupervised Feature Selection

Unsupervised feature selection for multi-cluster data

Unsupervised Feature Selection with Joint Clustering Analysis.

Hypergraph spectra for unsupervised feature selection

TCIC_FS: Total Correlation Information Coefficient-Based Feature Selection Method for High-Dimensional Data

Unsupervised Feature Selection Via Hypergraph Embedding

Feature Selection in Machine Learning: A New Perspective

Joint Cauchy Dictionary Learning and Graph Learning for Unsupervised Feature Selection

An unsupervised feature selection method based on degree of feature cooperation.

Discriminative and Uncorrelated Feature Selection with Constrained Spectral Analysis in Unsupervised Learning

Unsupervised Feature Selection Via Graph Matrix Learning and the Low-Dimensional Space Learning for Classification.