Abstract:Developing a robust deep neural network (DNN) for a specific task is not only time-consuming but also requires lots of experienced human experts. In order to make deep neural networks easier to apply or even take the human experts out of the design of network architecture completely, a growing number of researches focus on robust automated machine learning (AutoML). In this paper, we investigated the robustness problem of AutoML systems based on contractive pseudoinverse learners. In our proposed method, deep neural networks were built with stacked contractive pseudoinverse learners (CPILer). Each CPILer has a Jacobian regularized reconstruction loss function and is trained with pseudoinverse learning algorithm. When sigmoid activation function is adopted in the hidden layer, the graph Laplace regularizer is derived from square Frobenius norm of the Jacobian matrix. This learning scheme not only speeds up the training process dramatically but also reduces the effort of hyperparameter tuning. In addition, the graph Laplace regularization can improve the robustness of the learning systems by reducing the sensibility to noise. An ensemble network architecture consisting of several sub-networks was designed to build the AutoML systems. The architecture hyperparameters of the system were determined in an automated way which could be considered as a data-driven way. The proposed method shown good performance in the experiments in terms of efficiency and accuracy, and outperformed the baseline methods on a series of benchmark data sets. The robustness improvement of our proposed method was also demonstrated in the experiments.

A Vest of the Pseudoinverse Learning Algorithm.

Review of Pseudoinverse Learning Algorithm for Multilayer Neural Networks and Applications.

Pseudoinverse Learners - New Trend and Applications to Big Data.

A pseudoinverse learning algorithm for feedforward neural networks with stacked generalization applications to software reliability growth data

Boosting and Residual Learning Scheme with Pseudoinverse Learners

PILAE: A Non-gradient Descent Learning Scheme for Deep Feedforward Neural Networks

Bayesian Pseudoinverse Learners: From Uncertainty to Deterministic Learning

Pseudoinverse Learning Algorithom for Fast Sparse Autoencoder Training.

Kernel Principal Component Analysis Pseudoinverse Learning Algorithm

Autoencoder, Low Rank Approximation and Pseudoinverse Learning Algorithm.

DensePILAE: a feature reuse pseudoinverse learning algorithm for deep stacked autoencoder

A Robust Automated Machine Learning System with Pseudoinverse Learning.

Broad and Pseudoinverse Learning for Autoencoder

A Pseudoinversion-Free Method for Weight Updating in Broad Learning System

Pseudo inverse versus iterated projection: Novel learning approach and its application on broad learning system

A Pseudoinverse Incremental Algorithm For Fast Training Deep Neural Networks With Application To Spectra Pattern Recognition

A new approach to optical multilayer learning neural network

An Efficient and Effective Deep Convolutional Kernel Pseudoinverse Learner with Multi-Filter.

A Real Coded Population-Based Incremental Learning for Inverse Problems in Continuous Space

A Hierarchical Model with Pseudoinverse Learning Algorithm Optimazation for Pulsar Candidate Selection

Semi-adaptive Synergetic Two-way Pseudoinverse Learning System