Abstract:Deep ensemble learning models that combine multiple independent deep learning models with multi-layer processing architectures have proven to be effective techniques for improving the accuracy and robustness of deep learning models. In this paper, we propose a diversified kernel ensemble regression method, which is developed from the well-known kernel ridge regression methods. Motivated by multi-view data modeling ideas, we treat each individual kernel as one view of original data in kernel representation space. Therefore, we develop a deep kernel ensemble ridge regression method in Neural Tangent Kernel (NTK) to address the problem faced by traditional kernel ridge regression methods in finding appropriate types of kernels and their parameters. Specifically, as multiple deep kernel regressors share common information from the multi-view kernel representations, our proposed method is built through a hierarchical modeling method, where deep kernel regressors share a common parameter and also have model-specific parameters in individual regressors, which is further helpful in improving our model's performance. Furthermore, to achieve better diversified deep kernel representations in our proposed method, the Hilbert-Schmidt Independence Criterion (HSIC) is used to regularize our proposed model. In this way, we can find more diversified kernel representations among multiple kernel ensemble regressors to achieve better generalization performance. Experiments on several classification and regression datasets, such as MNIST, TinyImageNet-200, ORL, Credit and YearPredictionMSD demonstrate that our proposed method can achieve best regression performance than other state-of-the-art methods.

UA-DRN: Unbiased Aggregation of Deep Neural Networks for Regression Ensemble

UDRN: Unified Dimensional Reduction Neural Network for Feature Selection and Feature Projection

UDRN: Unified Dimensional Reduction Neural Network for Feature Selection and Feature Projection.

Dynamically Weighted Ensemble Neural Networks with Generalized Regression Neural Network for Solving Regression Problems

Dynamically Weighted Ensemble Neural Networks for Regression Problems

Efficient Diversity-Driven Ensemble for Deep Neural Networks

A DNN-Ensemble Method for Error Reduction and Training Data Selection in DNN Based Modeling

An Online Learning Neural Network Ensembles with Random Weights for Regression of Sequential Data Stream

RBUE: A ReLU-Based Uncertainty Estimation Method of Deep Neural Networks

Nonlinear Regression via Deep Negative Correlation Learning

Diversified deep hierarchical kernel ensemble regression

A hybrid ensemble method with negative correlation learning for regression

Combining Neural-Based Regression Predictors Using An Unbiased And Normalized Linear Ensemble Model

Robust Regression Via Deep Negative Correlation Learning

Why have a Unified Predictive Uncertainty? Disentangling it using Deep Split Ensembles

ERDeR: The Combination of Statistical Shrinkage Methods and Ensemble Approaches to Improve the Performance of Deep Regression

Lightweight Approaches to DNN Regression Error Reduction: an Uncertainty Alignment Perspective.

Deep Negative Correlation Classification

Randomized ReLU Activation for Uncertainty Estimation of Deep Neural Networks

Ensemble Deep Learning for Regression and Time Series Forecasting

The Diversified Ensemble Neural Network.