Abstract:Dimensional reduction~(DR) maps high-dimensional data into a lower dimensions latent space with minimized defined optimization objectives. The DR method usually falls into feature selection~(FS) and feature projection~(FP). FS focuses on selecting a critical subset of dimensions but risks destroying the data distribution (structure). On the other hand, FP combines all the input features into lower dimensions space, aiming to maintain the data structure; but lacks interpretability and sparsity. FS and FP are traditionally incompatible categories; thus, they have not been unified into an amicable framework. We propose that the ideal DR approach combines both FS and FP into a unified end-to-end manifold learning framework, simultaneously performing fundamental feature discovery while maintaining the intrinsic relationships between data samples in the latent space. In this work, we develop a unified framework, Unified Dimensional Reduction Neural-network~(UDRN), that integrates FS and FP in a compatible, end-to-end way. We improve the neural network structure by implementing FS and FP tasks separately using two stacked sub-networks. In addition, we designed data augmentation of the DR process to improve the generalization ability of the method when dealing with extensive feature datasets and designed loss functions that can cooperate with the data augmentation. Extensive experimental results on four image and four biological datasets, including very high-dimensional data, demonstrate the advantages of DRN over existing methods~(FS, FP, and FS\&FP pipeline), especially in downstream tasks such as classification and visualization.

Parameterless Reconstructive Discriminant Analysis for Feature Extraction

Discriminative analysis of direct linear discriminant analysis for face recognition

UDRN: Unified Dimensional Reduction Neural Network for Feature Selection and Feature Projection

Data-Dependent Kernel Discriminant Analysis for Feature Extraction and Classification

Efficient Kernel Discriminant Analysis via Spectral Regression

Locality-regularized Linear Regression Discriminant Analysis for Feature Extraction

Speed up kernel discriminant analysis

A Kernel Fractional-Step Nonlinear Discriminant Analysis for Pattern Recognition

Adaptive Linear Discriminant Regression Classification for Face Recognition

Nonparametric Locality Reforming Projection for Linear Discriminative Dimensionality Reduction

Collaborative Representation Based Local Discriminant Projection for Feature Extraction

SRDA: an Efficient Algorithm for Large-Scale Discriminant Analysis

Discriminative Principal Component Analysis: A REVERSE THINKING

Discriminant Local Feature Analysis with Applications to Face Recognition

Feature Extraction By Locality-Based Linear Discriminant Analysis

Constrained Linear Discrimination Analysis and Face Recognition

An Efficient Reformative Kernel Discriminant Analysis for Face Recognition

Adaptive quasiconformal kernel discriminant analysis

Local Feature Discriminant Projection

Linear Laplacian Discrimination for Feature Extraction

SRDA: an Efficient Algorithm for Large-Scale