Abstract:Dimensional reduction~(DR) maps high-dimensional data into a lower dimensions latent space with minimized defined optimization objectives. The DR method usually falls into feature selection~(FS) and feature projection~(FP). FS focuses on selecting a critical subset of dimensions but risks destroying the data distribution (structure). On the other hand, FP combines all the input features into lower dimensions space, aiming to maintain the data structure; but lacks interpretability and sparsity. FS and FP are traditionally incompatible categories; thus, they have not been unified into an amicable framework. We propose that the ideal DR approach combines both FS and FP into a unified end-to-end manifold learning framework, simultaneously performing fundamental feature discovery while maintaining the intrinsic relationships between data samples in the latent space. In this work, we develop a unified framework, Unified Dimensional Reduction Neural-network~(UDRN), that integrates FS and FP in a compatible, end-to-end way. We improve the neural network structure by implementing FS and FP tasks separately using two stacked sub-networks. In addition, we designed data augmentation of the DR process to improve the generalization ability of the method when dealing with extensive feature datasets and designed loss functions that can cooperate with the data augmentation. Extensive experimental results on four image and four biological datasets, including very high-dimensional data, demonstrate the advantages of DRN over existing methods~(FS, FP, and FS\&FP pipeline), especially in downstream tasks such as classification and visualization.

C2DNDA: A Deep Framework for Nonlinear Dimensionality Reduction

Deep Manifold Transformation for Dimension Reduction

Convolutional 2D LDA for Nonlinear Dimensionality Reduction

UDRN: Unified Dimensional Reduction Neural Network for Feature Selection and Feature Projection

Deep Manifold Transformation for Nonlinear Dimensionality Reduction

Deep Nonlinear Sufficient Dimension Reduction

A Local Similarity-Preserving Framework for Nonlinear Dimensionality Reduction with Neural Networks

Two-dimensional Joint Local and Nonlocal Discriminant Analysis-Based 2D Image Feature Extraction for Deep Learning

Nonlinear Dimensionality Reduction of Data by Deep Distributed Random Samplings.

Adaptive Local Linear Discriminant Analysis

Linear Dimensionality Reduction Based on Hybrid Structure Preserving Projections

Nonlinear Dimensionality Reduction with Relative Distance Comparison

Robust Two-Dimensional Capped L2, 1-Norm Linear Discriminant Analysis with Regularization and Its Applications on Image Recognition.

(2D)2LDALPP: A Novel Approach to Face Recognition

Optimal Dimensionality Discriminant Analysis and Its Application to Image Recognition

Locality Preserving Projection Via Deep Neural Network.

L1-norm Based Two-dimensional Linear Discriminant Analysis

Regularized Coplanar Discriminant Analysis for Dimensionality Reduction

A Novel Dimensionality Reduction Method with Discriminative Generalized Eigen-Decomposition.

Locality Adaptive Discriminant Analysis Framework

Distortion-Free Nonlinear Dimensionality Reduction