Abstract:Three layer feed-forward neural network (3-LFFNN) has been widely used for nonlinear regression. It is well known that its hidden layer can be regarded as taking the role of feature extraction and dimension reduction, and that the regression performance relies on how the feature dimension or equivalently the number of hidden units is determined appropriately. There are many publications on determining the hidden unit number for a desired generalization error. However, few comparative studies have been made on different approaches proposed, especially on those typical model selection criteria for this purpose. This paper targets such an aim. Using both simulated data and several real world data sets, a comparative study has been made on the regression performances with the number of hidden units determined by several typical model selection criteria, including Akaike’s Information Criterion (AIC), the consistent Akaike’s information criterion (CAIC), Schwarz’s Bayesian Inference Criterion (BIC) which coincides with Rissanen’s Minimum Description Length (MDL) criterion, and the well-known technique cross-validation (CV), as well as the Bayesian Ying-Yang harmony criterion on a small sample size (BYY-S). As shown in experiments on a small size of samples, BIC and CV are better than AIC and CAIC obviously. Moreover, BIC may be better than CV on certain data sets, while CV may be better than BIC on other data sets. Interestingly, BYY-S generally outperforms both BIC and CV.

Towards Efficient Dimensionality Reduction for Evolving Bayesian Network Classifier

Dimensionality reduction for evolving neural network

Data dimensionality reduction with application to simplifying RBF network structure and improving classification performance.

PAM: An Efficient Hybrid Dimension Reduction Algorithm for High-Dimensional Bayesian Network

Fusion of effective dimension reduction and discriminative dictionary learning for high-dimensional classification

An Efficient Dual Dimensionality Reduction Scheme of Features for Image Classification.

Data dimensionality reduction with application to improving classification performance and explaining concepts of data sets.

Dimension reduction in binary response regression: A joint modeling approach

Divergence Regulated Encoder Network for Joint Dimensionality Reduction and Classification

A Hybrid Method for Learning Multi-Dimensional Bayesian Network Classifiers Based on an Optimization Model

Efficient heuristics for learning scalable Bayesian network classifier from labeled and unlabeled data

Dimension reduction for covariates in network data

A Novel Approach to Fully Representing the Diversity in Conditional Dependencies for Learning Bayesian Network Classifier

Supervised Dimensionality Reduction for Multi-Dimensional Classification

A Parallel and Incremental Approach for Data-Intensive Learning of Bayesian Networks

Comparative investigation on dimension reduction and regression in three layer feed-forward neural network

Reducing Error of Tumor Classification by Using Dimension Reduction with Feature Selection

Boosting Simple Projections for Multi-Class Dimensionality Reduction

An Experimental Study of Dimension Reduction Methods on Machine Learning Algorithms with Applications to Psychometrics

Local logistic regression for dimension reduction in classification

Dynamic Modeling of Bayesian Classifier Based on General Information Theory