Abstract:Traditional Convolutional Neural Networks (CNNs) typically use the same activation function (usually ReLU) for all neurons with non-linear mapping operations. For example, the deep convolutional architecture Inception-v4 uses ReLU. To improve the classification performance of traditional CNNs, a new "Multi-function Convolutional Neural Network" (MCNN) is created by using different activation functions for different neurons. For $n$ neurons and $m$ different activation functions, there are a total of $m^n-m$ MCNNs and only $m$ traditional CNNs. Therefore, the best model is very likely to be chosen from MCNNs because there are $m^n-2m$ more MCNNs than traditional CNNs. For performance analysis, two different datasets for two applications (classifying handwritten digits from the MNIST database and classifying brain MRI images into one of the four stages of Alzheimer's disease (AD)) are used. For both applications, an activation function is randomly selected for each layer of a MCNN. For the AD diagnosis application, MCNNs using a newly created multi-function Inception-v4 architecture are constructed. Overall, simulations show that MCNNs can outperform traditional CNNs in terms of multi-class classification accuracy for both applications. An important future research work will be to efficiently select the best MCNN from $m^n-m$ candidate MCNNs. Current CNN software only provides users with partial functionality of MCNNs since different layers can use different activation functions but not individual neurons in the same layer. Thus, modifying current CNN software systems such as ResNets, DenseNets, and Dual Path Networks by using multiple activation functions and developing more effective and faster MCNN software systems and tools would be very useful to solve difficult practical image classification problems.

Invariant Feature Extraction for Image Classification Via Multi-Channel Convolutional Neural Network

Database Systems for Advanced Applications

Transform-Invariant Convolutional Neural Networks for Image Classification and Search

Rotation-Invariant Feature Learning in VHR Optical Remote Sensing Images Via Nested Siamese Structure with Double Center Loss.

Rotation-Invariant Feature Learning for Object Detection in VHR Optical Remote Sensing Images by Double-Net

Rotation-Invariant Feature Learning Via Convolutional Neural Network with Cyclic Polar Coordinates Convolutional Layer

Feature Learning For SAR Images Using Convolutional Neural Network.

Coarse-to-fine Trained Multi-Scale Convolutional Neural Networks for Image Classification.

Multi-Spectral RGB-NIR Image Classification Using Double-Channel CNN

Robust Multi-Classifier for Camera Model Identification Based on Convolution Neural Network

Group Invariant Deep Representations for Image Instance Retrieval

Multi-Channel and Multi-Scale Mid-Level Image Representation for Scene Classification

Affine Invariant Feature Extraction Based on Multi-scale Auto-convolution Entropy

3D Object Classification Based on Multi Convolutional Neural Networks

Multi-function Convolutional Neural Networks for Improving Image Classification Performance

Robust Neighborhood Preserving Low-Rank Sparse CNN Features for Classification.

An invariant feature extraction for multi-modal images matching

Multi-scale and Discriminative Part Detectors Based Features for Multi-label Image Classification.

Learning Geometric Invariance Features and Discrimination Representation for Image Classification via Spatial Transform Network and XGBoost Modeling

On the ability of CNNs to extract color invariant intensity based features for image classification

A New Invariant Feature for Multi-Modal Images Matching