Abstract:Dimensional reduction (DR) maps high-dimensional data into a lower dimensions latent space with minimized defined optimization objectives. The two independent branches of DR are feature selection (FS) and feature projection (FP). FS focuses on selecting a critical subset of dimensions but risks destroying the data distribution (structure). On the other hand, FP combines all the input features into lower dimensions space, aiming to maintain the data structure, but lacks interpretability and sparsity. Moreover, FS and FP are traditionally incompatible categories and have not been unified into an amicable framework. Therefore, we consider that the ideal DR approach combines both FS and FP into a unified end-to-end manifold learning framework, simultaneously performing fundamental feature discovery while maintaining the intrinsic relationships between data samples in the latent space. This paper proposes a unified framework named Unified Dimensional Reduction Network (UDRN) to integrate FS and FP in an end-to-end way. Furthermore, a novel network framework is designed to implement FS and FP tasks separately using a stacked feature selection network and feature projection network. In addition, a stronger manifold assumption and a novel loss function are proposed. Furthermore, the loss function can leverage the priors of data augmentation to enhance the generalization ability of the proposed UDRN. Finally, comprehensive experimental results on four image and four biological datasets, including very high-dimensional data, demonstrate the advantages of DRN over existing methods (FS, FP, and FS&FP pipeline), especially in downstream tasks such as classification and visualization.

A new deep neural network based on a stack of single-hidden-layer feedforward neural networks with randomly fixed hidden neurons

UDRN: Unified Dimensional Reduction Neural Network for Feature Selection and Feature Projection.

SVM-based Deep Stacking Networks

Stacked Broad Learning System: From Incremental Flatted Structure to Deep Model

Stacked unsupervised learning with a network architecture found by supervised meta-learning

Hybrid Deep Neural Network Based on SDAE and GRUNN

Stacking-based neural network for nonlinear time series analysis

Stacked Convolutional Denoising Auto-Encoders for Feature Representation

Rapid Feature Learning with Stacked Linear Denoisers

Stacked What-Where Auto-encoders

Stacked Convolutional Sparse Auto-Encoders for Representation Learning

Deep Stacking Networks for Low-Resource Chinese Word Segmentation with Transfer Learning

Stack Multiple Shallow Autoencoders into a Strong One: A New Reconstruction-Based Method to Detect Anomaly

Feature-Aligned Stacked Autoencoder: A Novel Semisupervised Deep Learning Model for Pattern Classification of Industrial Faults

A layer-stress learning framework universally augments deep neural network tasks

Learning Hierarchical Structures with Differentiable Nondeterministic Stacks

Sparse Deep Stacking Network for Image Classification

Denoising Auto-Encoders Toward Robust Unsupervised Feature Representation.

Tensor based stacked fuzzy neural network for efficient data regression

Stacking as Accelerated Gradient Descent

LORS: Low-rank Residual Structure for Parameter-Efficient Network Stacking