Abstract:Dimensional reduction (DR) maps high-dimensional data into a lower dimensions latent space with minimized defined optimization objectives. The two independent branches of DR are feature selection (FS) and feature projection (FP). FS focuses on selecting a critical subset of dimensions but risks destroying the data distribution (structure). On the other hand, FP combines all the input features into lower dimensions space, aiming to maintain the data structure, but lacks interpretability and sparsity. Moreover, FS and FP are traditionally incompatible categories and have not been unified into an amicable framework. Therefore, we consider that the ideal DR approach combines both FS and FP into a unified end-to-end manifold learning framework, simultaneously performing fundamental feature discovery while maintaining the intrinsic relationships between data samples in the latent space. This paper proposes a unified framework named Unified Dimensional Reduction Network (UDRN) to integrate FS and FP in an end-to-end way. Furthermore, a novel network framework is designed to implement FS and FP tasks separately using a stacked feature selection network and feature projection network. In addition, a stronger manifold assumption and a novel loss function are proposed. Furthermore, the loss function can leverage the priors of data augmentation to enhance the generalization ability of the proposed UDRN. Finally, comprehensive experimental results on four image and four biological datasets, including very high-dimensional data, demonstrate the advantages of DRN over existing methods (FS, FP, and FS&FP pipeline), especially in downstream tasks such as classification and visualization.

Canonical Kernel Dimension Reduction

UDRN: Unified Dimensional Reduction Neural Network for Feature Selection and Feature Projection

UDRN: Unified Dimensional Reduction Neural Network for Feature Selection and Feature Projection.

Self-representative kernel concept factorization

Sparse kernel sufficient dimension reduction

Distance-Based Classifier Via the Kernel Trick.

Learning Linear Representation of Space Partitioning Trees Based on Unsupervised Kernel Dimension Reduction

Data-Dependent Kernel Discriminant Analysis for Feature Extraction and Classification

Kernel optimization-based discriminant analysis for face recognition

Kernel alignment unsupervised discriminative dimensionality reduction

Speed up kernel discriminant analysis

New forest-based approaches for sufficient dimension reduction

Multi-layer Cross-domain Non-negative Matrix Factorization for Cross-scene Dimension Reduction on Hyperspectral Images.

Supervised Dimensionality Reduction Technology of Generalized Discriminant Component Analysis and Its Kernelization Forms

Graph kernel by spectral transform in kernel fisher discriminant analysis

Robust sufficient dimension reduction via α-distance covariance

Kernelized Sparse Graph-Embedded Dimensionality Reduction for Hyperspectral Image Classification.

FUNCTIONAL SUFFICIENT DIMENSION REDUCTION THROUGH AVERAGE FRÉCHET DERIVATIVES

Scalable Dual Coordinate Descent for Kernel Methods

Kernel distance measures for time series, random fields and other structured data

Online Kernel Sliced Inverse Regression