Abstract:Traditional Convolutional Neural Networks (CNNs) typically use the same activation function (usually ReLU) for all neurons with non-linear mapping operations. For example, the deep convolutional architecture Inception-v4 uses ReLU. To improve the classification performance of traditional CNNs, a new "Multi-function Convolutional Neural Network" (MCNN) is created by using different activation functions for different neurons. For $n$ neurons and $m$ different activation functions, there are a total of $m^n-m$ MCNNs and only $m$ traditional CNNs. Therefore, the best model is very likely to be chosen from MCNNs because there are $m^n-2m$ more MCNNs than traditional CNNs. For performance analysis, two different datasets for two applications (classifying handwritten digits from the MNIST database and classifying brain MRI images into one of the four stages of Alzheimer's disease (AD)) are used. For both applications, an activation function is randomly selected for each layer of a MCNN. For the AD diagnosis application, MCNNs using a newly created multi-function Inception-v4 architecture are constructed. Overall, simulations show that MCNNs can outperform traditional CNNs in terms of multi-class classification accuracy for both applications. An important future research work will be to efficiently select the best MCNN from $m^n-m$ candidate MCNNs. Current CNN software only provides users with partial functionality of MCNNs since different layers can use different activation functions but not individual neurons in the same layer. Thus, modifying current CNN software systems such as ResNets, DenseNets, and Dual Path Networks by using multiple activation functions and developing more effective and faster MCNN software systems and tools would be very useful to solve difficult practical image classification problems.

Convolutional Neural Network Incorporating Misclassification Information for Image Recognition

Research on Image Classification Method of Features of Combinatorial Convolution

HCFNN: High-order Coverage Function Neural Network for Image Classification

Image Classification Method Rationally Utilizing Spatial Information of the Image

A new hybrid model of convolutional neural networks and hidden Markov chains for image classification

Applying Improved Convolutional Neural Network in Image Classification

A Study on CNN Transfer Learning for Image Classification

Simple convolutional neural network on image classification

Transform-Invariant Convolutional Neural Networks for Image Classification and Search

Image Target Recognition Based on Improved Convolutional Neural Network

Multi-function Convolutional Neural Networks for Improving Image Classification Performance

Coarse-to-fine Trained Multi-Scale Convolutional Neural Networks for Image Classification.

Midcn: A Multiple Instance Deep Convolutional Network For Image Classification

Class attention network for image recognition

Wasserstein Pooling for Image Classification

Training Interpretable Convolutional Neural Networks by Differentiating Class-specific Filters

Towards Learning Spatially Discriminative Feature Representations

Convolutional Networks with Cross-Layer Neurons for Image Recognition

Reconstruction Combined Training for Convolutional Neural Networks on Character Recognition

An Image Classification Method Based on Adaptive Attention Mechanism and Feature Extraction Network

Learning Geometric Invariance Features and Discrimination Representation for Image Classification via Spatial Transform Network and XGBoost Modeling