Abstract:Developing a robust deep neural network (DNN) for a specific task is not only time-consuming but also requires lots of experienced human experts. In order to make deep neural networks easier to apply or even take the human experts out of the design of network architecture completely, a growing number of researches focus on robust automated machine learning (AutoML). In this paper, we investigated the robustness problem of AutoML systems based on contractive pseudoinverse learners. In our proposed method, deep neural networks were built with stacked contractive pseudoinverse learners (CPILer). Each CPILer has a Jacobian regularized reconstruction loss function and is trained with pseudoinverse learning algorithm. When sigmoid activation function is adopted in the hidden layer, the graph Laplace regularizer is derived from square Frobenius norm of the Jacobian matrix. This learning scheme not only speeds up the training process dramatically but also reduces the effort of hyperparameter tuning. In addition, the graph Laplace regularization can improve the robustness of the learning systems by reducing the sensibility to noise. An ensemble network architecture consisting of several sub-networks was designed to build the AutoML systems. The architecture hyperparameters of the system were determined in an automated way which could be considered as a data-driven way. The proposed method shown good performance in the experiments in terms of efficiency and accuracy, and outperformed the baseline methods on a series of benchmark data sets. The robustness improvement of our proposed method was also demonstrated in the experiments.

Pseudoinverse Learners - New Trend and Applications to Big Data.

Review of Pseudoinverse Learning Algorithm for Multilayer Neural Networks and Applications.

A Vest of the Pseudoinverse Learning Algorithm.

Bayesian Pseudoinverse Learners: From Uncertainty to Deterministic Learning

A pseudoinverse learning algorithm for feedforward neural networks with stacked generalization applications to software reliability growth data

Boosting and Residual Learning Scheme with Pseudoinverse Learners

Pseudoinverse Learning Algorithom for Fast Sparse Autoencoder Training.

Broad and Pseudoinverse Learning for Autoencoder

DensePILAE: a feature reuse pseudoinverse learning algorithm for deep stacked autoencoder

Kernel Principal Component Analysis Pseudoinverse Learning Algorithm

PILAE: A Non-gradient Descent Learning Scheme for Deep Feedforward Neural Networks

A Robust Automated Machine Learning System with Pseudoinverse Learning.

An Efficient and Effective Deep Convolutional Kernel Pseudoinverse Learner with Multi-Filter.

Autoencoder, Low Rank Approximation and Pseudoinverse Learning Algorithm.

A Real Coded Population-Based Incremental Learning for Inverse Problems in Continuous Space

Pseudo inverse versus iterated projection: Novel learning approach and its application on broad learning system

Image Recognition Based on Combined Filters with Pseudoinverse Learning Algorithm.

A Pseudoinverse Incremental Algorithm For Fast Training Deep Neural Networks With Application To Spectra Pattern Recognition

Semi-adaptive Synergetic Two-way Pseudoinverse Learning System

Diverse Power Iteration Embeddings: Theory and Practice

A Spatial-Temporally Adaptive PINN Framework for 3D Bi-Ventricular Electrophysiological Simulations and Parameter Inference