Abstract:Deep learning is a kind of feature learning method with strong nonliear feature transformation and becomes more and more important in many fields of artificial intelligence. Deep autoencoder is one representative method of the deep learning methods, and can effectively extract abstract the information of datasets. However, it does not consider the complementarity between the deep features and original features during deep feature transformation. Besides, it suffers from small sample problem. In order to solve these problems, a novel deep autoencoder - hybrid feature embedded stacked sparse autoencoder(HESSAE) has been proposed in this paper. HFESAE is capable to learn discriminant deep features with the help of embedding original features to filter weak hidden-layer outputs during training. For the issue that class representation ability of abstract information is limited by small sample problem, a feature fusion strategy has been designed aiming to combining abstract information learned by HFESAE with original feature and obtain hybrid features for feature reduction. The strategy is hybrid feature selection strategy based on L1 regularization followed by an support vector machine(SVM) ensemble model, in which weighted local discriminant preservation projection (w_LPPD), is designed and employed on each base classifier. At the end of this paper, several representative public datasets are used to verify the effectiveness of the proposed algorithm. The experimental results demonstrated that, the proposed feature learning method yields superior performance compared to other existing and state of art feature learning algorithms including some representative deep autoencoder methods.

Structured Data Encoder for Neural Networks Based on Gradient Boosting Decision Tree.

Gradient-Boosted Based Structured and Unstructured Learning

An Automated Data Mining Framework Using Autoencoders for Feature Extraction and Dimensionality Reduction

StructBoost: Boosting Methods for Predicting Structured Output Variables

Feature Encodings for Gradient Boosting with Automunge

Embeddings and Representation Learning for Structured Data

Exploiting Problem Structure in Deep Declarative Networks: Two Case Studies

An Architecture as an Alternative to Gradient Boosted Decision Trees for Multiple Machine Learning Tasks

StructureBoost: Efficient Gradient Boosting for Structured Categorical Variables

Features that Make a Difference: Leveraging Gradients for Improved Dictionary Learning

Multi-Layered Gradient Boosting Decision Trees.

Denoising Auto-Encoders Toward Robust Unsupervised Feature Representation.

A generalized decision tree ensemble based on the NeuralNetworks architecture: Distributed Gradient Boosting Forest (DGBF)

Structural Optimization Makes Graph Classification Simpler and Better

Structured IB: Improving Information Bottleneck with Structured Feature Learning

Multi-grained and multi-layered gradient boosting decision tree for credit scoring

Beyond Deep Learning: An Evolutionary Feature Engineering Approach to Tabular Data Classification

Deep Boosting: Joint Feature Selection and Analysis Dictionary Learning in Hierarchy

Deep Graph Convolutional Encoders for Structured Data to Text Generation

Hybrid Embedded Deep Stacked Sparse Autoencoder with w_LPPD SVM Ensemble

Network Embedding via Deep Prediction Model