Abstract:Feature selection (FS) is an important technique in machine learning and data mining, especially for large-scale high-dimensional data. Most existing studies have been restricted to batch learning, which is often inefficient and poorly scalable when handling big data in real world. As real data may arrive sequentially and continuously, batch learning has to retrain the model for the new coming data, which is very computationally intensive. Online feature selection (OFS) is a promising new paradigm that is more efficient and scalable than batch learning algorithms. However, existing online algorithms usually fall short in their inferior efficacy. In this article, we present a novel second-order OFS algorithm that is simple yet effective, very fast and extremely scalable to deal with large-scale ultra-high dimensional sparse data streams. The basic idea is to exploit the second-order information to choose the subset of important features with high confidence weights. Unlike existing OFS methods that often suffer from extra high computational cost, we devise a novel algorithm with a MaxHeap-based approach, which is not only more effective than the existing first-order algorithms, but also significantly more efficient and scalable. Our extensive experiments validated that the proposed technique achieves highly competitive accuracy as compared with state-of-the-art batch FS methods, meanwhile it consumes significantly less computational cost that is orders of magnitude lower. Impressively, on a billion-scale synthetic dataset (1-billion dimensions, 1-billion non-zero features, and 1-million samples), the proposed algorithm takes less than 3 minutes to run on a single PC.

OFS-Density: A novel online streaming feature selection method.

Online streaming feature selection using adapted Neighborhood Rough Set

Large-Scale Online Feature Selection for Ultra-High Dimensional Sparse Data.

Online Feature Selection for High-Dimensional Class-Imbalanced Data

U^2F^2S^2 : Uncovering Feature-level Similarities for Unsupervised Feature Selection

LOFS: A Library of Online Streaming Feature Selection.

LOFS: Library of Online Streaming Feature Selection

Online Streaming Feature Selection Via Conditional Independence

Online Feature Selection for Streaming Features Using Self-Adaption Sliding-Window Sampling

Unsupervised feature selection via dual space-based low redundancy scores and extended OLSDA

Online Feature Selection for Streaming Features with High Redundancy Using Sliding-Window Sampling

Feature Selection Based on Intrusive Outliers Rather Than All Instances

Online early terminated streaming feature selection based on Rough Set theory

Online Heterogeneous Streaming Feature Selection without Feature Type Information

Online Scalable Streaming Feature Selection Via Dynamic Decision

Adaptive Sparse Confidence-Weighted Learning for Online Feature Selection.

Local Feature Discriminant Projection

DPC-FSC: an Approach of Fuzzy Semantic Cells to Density Peaks Clustering

Fair Streaming Feature Selection

General Assembly Framework for Online Streaming Feature Selection Via Rough Set Models

Online Multi-Label Streaming Feature Selection Based on Label Group Correlation and Feature Interaction