Abstract:A critical aspect of dimensionality reduction is to assess the quality of selected (or produced) feature subsets properly. Feature subset assessment in machine learning refers to split a given feature subset into a training set, which is used to estimate the parameters of a classification model, and a test set used to estimate the predictive performance of the model. Then, averaging the results of multiple splitting (i.e., Cross-Validation, CV) is commonly used to decrease the variance of the estimator. But in practice, CV scheme is very computationally expensive. In this paper, we propose a new statistics index method called LW-index for evaluation of feature subset and dimensionality reduction algorithms in general. The proposed method is a type of “classical statistics” approach that uses the feature subset to compute an empirical estimate of the quality of feature subset. A large number of performance comparisons with the machine learning approach conducted on fourteen benchmark collections show that the proposed LW-index is highly correlated with the external indices (i.e., MacroF1, MicroF1) of SVM and Centroid-Based Classifier (CBC) trained by five-fold CV scheme. Furthermore, the experimental results indicate that LW-index has the same performance as the traditional CV scheme for evaluating the dimensionality reduction algorithms and it is more efficient than the traditional methodology. Therefore, one contribution of this paper is to present an alternative methodology, based on an internal index typically used in the unsupervised learning context, that is computationally cheaper than the traditional CV methodology. Another contribution is to propose a new internal index that behaves better than other similar indices widely used in clustering and shows high correlation with the results obtained by the traditional methodology.

Internal Purity: A Differential Entropy Based Internal Validation Index for Crisp and Fuzzy Clustering Validation

Internal Purity: A Differential Entropy based Internal Validation Index for Clustering Validation

A New Validity Index Based on Intra-Cluster Variation and Inter-Cluster Overlap

A GROUP OF NEW INDEXES OF CLUSTER VALIDITY

On the Index of Cluster Validity

Improved Validation Index for Fuzzy Clustering

A New Validity Index Based on Fuzzy Energy and Fuzzy Entropy Measures in Fuzzy Clustering Problems

CVAP: Validation for Cluster Analyses

A New Validity Index for Fuzzy-Possibilistic C-Means Clustering

A Fuzzy Cluster Validity Index Based on Clustering Mistake Measures

A New Index for Clustering Evaluation Based on Density Estimation

Quantitative evaluation of internal cluster validation indices using binary data sets

Efficient synthetical clustering validity indexes for hierarchical clustering

A new approach for evaluating internal cluster validation indices

Deep Clustering Evaluation: How to Validate Internal Clustering Validation Measures

An Information-Theoretic External Cluster-Validity Measure

External validation measures for K-means clustering

A new validity measure for fuzzy c-means clustering

From A-to-Z Review of Clustering Validation Indices

Validation of Overlapping Clustering: A Random Clustering Perspective

A New Validity Index of Feature Subset for Evaluating the Dimensionality Reduction Algorithms