Abstract:In this paper, we propose a polynomial kernel neural network classifier (PKNNC) based on the random sampling and information gain. Random sampling is used here to generate datasets for the construction of polynomial neurons located in the neural networks, while information gain is used to evaluate the importance of the input variables (viz. dataset features) of each neuron. Both random sampling and information gain stem from the concepts of well-known random forest models. Some traditional neural networks have certain limitations, such as slow convergence speed, easily falling to local optima and difficulty describing the polynomial relation between the input and output. In this regard, a general PKNNC is proposed, and it consists of three parts: the premise, conclusion, and aggregation. The method of designing the PKNNC is summarized as follows. In the premise section, random sampling and information gain are used to obtain multiple subdatasets that are passed to the aggregation part, and the conclusion part uses three types of polynomials. In the aggregation part, the least squares method (LSM) is used to estimate the parameters of polynomials. Furthermore, the particle swarm optimization (PSO) algorithm is exploited here to optimize the PKNNC. The overall optimization of the PKNNC combines structure optimization and parameter optimization. The PKNNC takes advantage of three types of polynomial kernel functions, random sampling techniques and information gain algorithms, which have a good ability to describe the higher-order nonlinear relationships between input and output variables and have high generalization and fast convergence capabilities. To evaluate the effectiveness of the PKNNC, numerical experiments are carried out on two types of data: machine learning data and face data. A comparative study illustrates that the proposed PKNNC leads to better performance than several conventional models.

Learning Deep Neural Network Based Kernel Functions for Small Sample Size Classification

DWS-MKL: Depth-width-scaling multiple kernel learning for data classification

Learning Explicit Deep Representations from Deep Kernel Networks

Optimizing Kernel Machines using Deep Learning

Deep Latent-Variable Kernel Learning

Guided Deep Kernel Learning

How to Scale Up Kernel Methods to Be As Good As Deep Neural Nets

Kernel optimization-based discriminant analysis for face recognition

Learning Deep Kernels for Non-Parametric Two-Sample Tests

Efficient kernel surrogates for neural network-based regression

Kernel Generalized Nonlinear Discriminant Analysis Algorithm For Pattern Recognition

Deep InterBoost networks for small-sample image classification

A New Method of Deep Convolutional Neural Network Image Classification Based on Knowledge Transfer in Small Label Sample Environment

Effect of Kernel Size in Deep Learning-Based Convolutional Neural Networks for Image Classification

Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning

An Optimized Neural Network Classification Method Based on Kernel Holistic Learning and Division

A polynomial kernel neural network classifier based on random sampling and information gain

Can Infinitely Wide Deep Nets Help Small-data Multi-label Learning?

Gauss-like Logarithmic Kernel Function to improve the performance of kernel machines on the small datasets

When Do Neural Networks Outperform Kernel Methods?

Two-sample Testing Using Deep Learning