Abstract:Recent advances in Convolutional Neural Networks (CNNs) have obtained promising results in difficult deep learning tasks. However, the success of a CNN depends on finding an architecture to fit a given problem. A hand-crafted architecture is a challenging, time-consuming process that requires expert knowledge and effort, due to a large number of architectural design choices. In this article, we present an efficient framework that automatically designs a high-performing CNN architecture for a given problem. In this framework, we introduce a new optimization objective function that combines the error rate and the information learnt by a set of feature maps using deconvolutional networks (deconvnet). The new objective function allows the hyperparameters of the CNN architecture to be optimized in a way that enhances the performance by guiding the CNN through better visualization of learnt features via deconvnet. The actual optimization of the objective function is carried out via the Nelder-Mead Method (NMM). Further, our new objective function results in much faster convergence towards a better architecture. The proposed framework has the ability to explore a CNN architecture’s numerous design choices in an efficient way and also allows effective, distributed execution and synchronization via web services. Empirically, we demonstrate that the CNN architecture designed with our approach outperforms several existing approaches in terms of its error rate. Our results are also competitive with state-of-the-art results on the MNIST dataset and perform reasonably against the state-of-the-art results on CIFAR-10 and CIFAR-100 datasets. Our approach has a significant role in increasing the depth, reducing the size of strides, and constraining some convolutional layers not followed by pooling layers in order to find a CNN architecture that produces a high recognition performance.

Improving CNN Performance with Min-Max Objective

Improving CNN Performance Accuracies with Min-Max Objective

Improving Deep Neural Network Performance by Integrating Kernelized Min-Max Objective

Improving object detection with deep convolutional networks via Bayesian optimization and structured prediction

MaxQ: Multi-Axis Query for N:M Sparsity Network

Integrating Supervised Laplacian Objective with CNN for Object Recognition.

Best Practices for Convolutional Neural Networks Applied to Object Recognition in Images

Optimal Hyperparameters and Structure Setting of Multi-Objective Robust CNN Systems via Generalized Taguchi Method and Objective Vector Norm

A improved pooling method for convolutional neural networks

Optimizing Convolutional Neural Network Architecture

Improving DCNN Performance with Sparse Category-Selective Objective Function.

An Improved CNN Model for Fast Salient Object Detection

Max Feature Map CNN with Support Vector Guided Softmax for Face Recognition

A Framework for Designing the Architectures of Deep Convolutional Neural Networks

Receptive Field Refinement for Convolutional Neural Networks Reliably Improves Predictive Performance

High-speed hyperparameter optimization for deep ResNet models in image recognition

COMO: Efficient Deep Neural Networks Expansion With COnvolutional MaxOut

Optimization of CNN through Novel Training Strategy for Visual Classification Problems

Towards General Robustness Verification of MaxPool-based Convolutional Neural Networks via Tightening Linear Approximation

Improved fine-grained object retrieval with Hard Global Softmin Loss objective

The Investigation of Multiple Optimization Methods on Convolutional Neural Network