Abstract:Recent advances in Convolutional Neural Networks (CNNs) have obtained promising results in difficult deep learning tasks. However, the success of a CNN depends on finding an architecture to fit a given problem. A hand-crafted architecture is a challenging, time-consuming process that requires expert knowledge and effort, due to a large number of architectural design choices. In this article, we present an efficient framework that automatically designs a high-performing CNN architecture for a given problem. In this framework, we introduce a new optimization objective function that combines the error rate and the information learnt by a set of feature maps using deconvolutional networks (deconvnet). The new objective function allows the hyperparameters of the CNN architecture to be optimized in a way that enhances the performance by guiding the CNN through better visualization of learnt features via deconvnet. The actual optimization of the objective function is carried out via the Nelder-Mead Method (NMM). Further, our new objective function results in much faster convergence towards a better architecture. The proposed framework has the ability to explore a CNN architecture’s numerous design choices in an efficient way and also allows effective, distributed execution and synchronization via web services. Empirically, we demonstrate that the CNN architecture designed with our approach outperforms several existing approaches in terms of its error rate. Our results are also competitive with state-of-the-art results on the MNIST dataset and perform reasonably against the state-of-the-art results on CIFAR-10 and CIFAR-100 datasets. Our approach has a significant role in increasing the depth, reducing the size of strides, and constraining some convolutional layers not followed by pooling layers in order to find a CNN architecture that produces a high recognition performance.

Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias

Convergence Analysis for Deep Sparse Coding via Convolutional Neural Networks

A Framework for Designing the Architectures of Deep Convolutional Neural Networks

Refining Architectures of Deep Convolutional Neural Networks

Exploring the Design Space of Deep Convolutional Neural Networks at Large Scale

Theoretical Analysis of Inductive Biases in Deep Convolutional Networks

From Compass and Ruler to Convolution and Nonlinearity: On the Surprising Difficulty of Understanding a Simple CNN Solving a Simple Geometric Estimation Task

On the Learning Dynamics of Two-layer Nonlinear Convolutional Neural Networks.

Low-Cost Parameterizations of Deep Convolutional Neural Networks

Make Deep Networks Shallow Again

Convolutional Neural Networks Analyzed via Inverse Problem Theory and Sparse Representations

The Heterogeneity Hypothesis: Finding Layer-Wise Differentiated Network Architectures

Convolutional Neural Networks Exploiting Attributes of Biological Neurons

Enhancing Convolutional Neural Networks with Higher-Order Numerical Difference Methods

When Face Recognition Meets with Deep Learning: An Evaluation of Convolutional Neural Networks for Face Recognition

Architecture evolution of convolutional neural network using monarch butterfly optimization

Receptive Field Refinement for Convolutional Neural Networks Reliably Improves Predictive Performance

On the importance of network architecture in training very deep neural networks

A Graph-Based Encoding for Evolutionary Convolutional Neural Network Architecture Design

Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators

Linking convolutional kernel size to generalization bias in face analysis CNNs