Abstract:The choice of parameters, and the design of the network architecture are important factors affecting the performance of deep neural networks. Genetic Algorithms (GA) have been used before to determine parameters of a network. Yet, GAs perform a finite search over a discrete set of pre-defined candidates, and cannot, in general, generate unseen configurations. In this paper, to move from exploration to exploitation, we propose a novel and systematic method that autonomously and simultaneously optimizes multiple parameters of any deep neural network by using a GA aided by a bi-generative adversarial network (Bi-GAN). The proposed Bi-GAN allows the autonomous exploitation and choice of the number of neurons, for fully-connected layers, and number of filters, for convolutional layers, from a large range of values. Our proposed Bi-GAN involves two generators, and two different models compete and improve each other progressively with a GAN-based strategy to optimize the networks during GA evolution. Our proposed approach can be used to autonomously refine the number of convolutional layers and dense layers, number and size of kernels, and the number of neurons for the dense layers; choose the type of the activation function; and decide whether to use dropout and batch normalization or not, to improve the accuracy of different deep neural network architectures. Without loss of generality, the proposed method has been tested with the ModelNet database, and compared with the 3D Shapenets and two GA-only methods. The results show that the presented approach can simultaneously and successfully optimize multiple neural network parameters, and achieve higher accuracy even with shallower networks.

Improving OPT-GAN by Smooth Scale Mapping and Adaptive Exploration

OPT-GAN: A Broad-Spectrum Global Optimizer for Black-box Problems by Learning Distribution

GL-GAN: Adaptive Global and Local Bilevel Optimization for Generative Adversarial Network

GL-GAN: Adaptive Global and Local Bilevel Optimization model of Image Generation

Manifold Interpolation for Large-Scale Multiobjective Optimization Via Generative Adversarial Networks

Manifold Interpolation for Large-Scale Multi-Objective Optimization via Generative Adversarial Networks

Improving GAN Training via Feature Space Shrinkage

Evolutionary Multiobjective Optimization Driven by Generative Adversarial Networks (Gans)

A Closer Look at the Optimization Landscapes of Generative Adversarial Networks

A GAN-based genetic algorithm for solving the 3D bin packing problem

GEGAN: Generative Adversarial Networks Training Based on Gradient Directed Evolutionary Strategy

Autonomously and Simultaneously Refining Deep Neural Network Parameters by a Bi-Generative Adversarial Network Aided Genetic Algorithm

Generative Evolutionary Strategy For Black-Box Optimizations

Data-Driven Structural Topology Optimization Method Using Conditional Wasserstein Generative Adversarial Networks with Gradient Penalty

On the Convergence and Robustness of Training GANs with Regularized Optimal Transport

MCGAN: Enhancing GAN Training with Regression-Based Generator Loss

Improving GAN with Neighbors Embedding and Gradient Matching

Offline Data-Driven Multi-objective Optimization Evolutionary Algorithm based on Generative Adversarial Network

Optimization-Inspired Learning with Architecture Augmentations and Control Mechanisms for Low-Level Vision

BGADAM: Boosting based Genetic-Evolutionary ADAM for Neural Network Optimization

Genetically Modified Wolf Optimization with Stochastic Gradient Descent for Optimising Deep Neural Networks