Abstract:The choice of parameters, and the design of the network architecture are important factors affecting the performance of deep neural networks. Genetic Algorithms (GA) have been used before to determine parameters of a network. Yet, GAs perform a finite search over a discrete set of pre-defined candidates, and cannot, in general, generate unseen configurations. In this paper, to move from exploration to exploitation, we propose a novel and systematic method that autonomously and simultaneously optimizes multiple parameters of any deep neural network by using a GA aided by a bi-generative adversarial network (Bi-GAN). The proposed Bi-GAN allows the autonomous exploitation and choice of the number of neurons, for fully-connected layers, and number of filters, for convolutional layers, from a large range of values. Our proposed Bi-GAN involves two generators, and two different models compete and improve each other progressively with a GAN-based strategy to optimize the networks during GA evolution. Our proposed approach can be used to autonomously refine the number of convolutional layers and dense layers, number and size of kernels, and the number of neurons for the dense layers; choose the type of the activation function; and decide whether to use dropout and batch normalization or not, to improve the accuracy of different deep neural network architectures. Without loss of generality, the proposed method has been tested with the ModelNet database, and compared with the 3D Shapenets and two GA-only methods. The results show that the presented approach can simultaneously and successfully optimize multiple neural network parameters, and achieve higher accuracy even with shallower networks.

Genetic algorithm for neural network architecture optimization

Compute-Efficient Neural Network Architecture Optimization by a Genetic Algorithm

Deep Genetic Network

An efficient optimization approach for designing machine learning models based on genetic algorithm

Optimizing Feed-Forward Neural Networks Using Cascaded Genetic Algorithm

Extending MLP ANN hyper-parameters Optimization by using Genetic Algorithm

Genetic Network Architecture Search

Designing Convolutional Neural Network Architecture Using Genetic Algorithms

Neural Networks Designing Neural Networks: Multi-Objective Hyper-Parameter Optimization

Optimization Study of BP Neural Network Based on Genetic Algorithm

An algorithmic framework for the optimization of deep neural networks architectures and hyperparameters

MCTS guided Genetic Algorithm for optimization of neural network weights

Autonomously and Simultaneously Refining Deep Neural Network Parameters by a Bi-Generative Adversarial Network Aided Genetic Algorithm

A Hybrid Genetic Algorithm-Neural Network Strategy for Simulation Optimization.

Neural Architecture Search Based on A Multi-objective Evolutionary Algorithm With Probability Stack

Genetic Algorithm Based Deep Learning Neural Network Structure and Hyperparameter Optimization

Self-learning Genetic Algorithm for Neural Network Topology Optimization

BGADAM: Boosting based Genetic-Evolutionary ADAM for Neural Network Optimization

An effective algorithm for hyperparameter optimization of neural networks

An Aggressive Genetic Programming Approach for Searching Neural Network Structure Under Computational Constraints