A 32-year-old man with recurrent cholangitis

G. Sandle,J. Lodge

DOI: https://doi.org/10.1016/S0140-6736(97)06172-2

IF: 202.731

1997-08-09

The Lancet

Abstract:

What problem does this paper attempt to address?

Enhanced Gradient for Differentiable Architecture Search

Haichao Zhang,Kuangrong Hao,Lei Gao,Xuesong Tang,Bing Wei

DOI: https://doi.org/10.48550/arXiv.2103.12529

2021-03-23

Abstract:In recent years, neural architecture search (NAS) methods have been proposed for the automatic generation of task-oriented network architecture in image classification. However, the architectures obtained by existing NAS approaches are optimized only for classification performance and do not adapt to devices with limited computational resources. To address this challenge, we propose a neural network architecture search algorithm aiming to simultaneously improve network performance (e.g., classification accuracy) and reduce network complexity. The proposed framework automatically builds the network architecture at two stages: block-level search and network-level search. At the stage of block-level search, a relaxation method based on the gradient is proposed, using an enhanced gradient to design high-performance and low-complexity blocks. At the stage of network-level search, we apply an evolutionary multi-objective algorithm to complete the automatic design from blocks to the target network. The experiment results demonstrate that our method outperforms all evaluated hand-crafted networks in image classification, with an error rate of on CIFAR10 and an error rate of on CIFAR100, both at network parameter size less than one megabit. Moreover, compared with other neural architecture search methods, our method offers a tremendous reduction in designed network architecture parameters.

Computer Vision and Pattern Recognition
G-EvoNAS: Evolutionary Neural Architecture Search Based on Network Growth

Juan Zou,Weiwei Jiang,Yizhang Xia,Yuan Liu,Zhanglu Hou

2024-03-05

Abstract:The evolutionary paradigm has been successfully applied to neural network search(NAS) in recent years. Due to the vast search complexity of the global space, current research mainly seeks to repeatedly stack partial architectures to build the entire model or to seek the entire model based on manually designed benchmark modules. The above two methods are attempts to reduce the search difficulty by narrowing the search space. To efficiently search network architecture in the global space, this paper proposes another solution, namely a computationally efficient neural architecture evolutionary search framework based on network growth (G-EvoNAS). The complete network is obtained by gradually deepening different Blocks. The process begins from a shallow network, grows and evolves, and gradually deepens into a complete network, reducing the search complexity in the global space. Then, to improve the ranking accuracy of the network, we reduce the weight coupling of each network in the SuperNet by pruning the SuperNet according to elite groups at different growth stages. The G-EvoNAS is tested on three commonly used image classification datasets, CIFAR10, CIFAR100, and ImageNet, and compared with various state-of-the-art algorithms, including hand-designed networks and NAS networks. Experimental results demonstrate that G-EvoNAS can find a neural network architecture comparable to state-of-the-art designs in 0.2 GPU days.

Neural and Evolutionary Computing
Neural Architecture Search with Progressive Evaluation and Sub-Population Preservation

Yu Xue,Jiajie Zha,Danilo Pelusi,Peng Chen,Tao Luo,Liangli Zhen,Yan Wang,Mohamed Wahib

DOI: https://doi.org/10.1109/tevc.2024.3393304

IF: 16.497

2024-01-01

IEEE Transactions on Evolutionary Computation

Abstract:Neural architecture search (NAS) is an effective approach for automating the design of deep neural networks. Evolutionary computation (EC) is commonly used in NAS due to its global optimization capability. However, the evaluation phase of architecture candidates in EC-based NAS is compute-intensive, limiting its application for many real-world problems. To overcome this challenge, we propose a novel progressive evaluation strategy for the evaluation phase in convolutional neural network architecture search, in which the number of training epochs of network individuals is progressively increased. In addition, a sub-population preservation strategy is proposed to preserve medium-size and large-size architectures to avoid prematurely discarding networks that may not perform well in the early stages but have the potential to excel with further optimization. Our proposed algorithm reduces the computational cost of the evaluation phase and promotes population diversity and fairness by preserving promising networks based on their distribution. We evaluate the proposed progressive evaluation and sub-population preservation of neural architecture search (PEPNAS) algorithm on the CIFAR10, CIFAR100, and ImageNet benchmark datasets, and compare it with 36 state-of-the-art algorithms, including manually designed networks, reinforcement learning (RL) algorithms, gradient-based algorithms, and other EC-based ones. The experimental results demonstrate that PEPNAS effectively identifies networks with competitive accuracy while also markedly improving the efficiency of the search process. For instance, PEPNAS discovers the architecture on CIFAR10 with a low error rate of 2.38% using only 0.7 GPU days. We directly adopt the searched architecture for the image classification on the CIFAR100 and ImageNet datasets, which achieves the top 1 error rates of 16.46% and 26.25%, respectively. The code is available at https://github.com/chajiajie/PEPNAS.
Neural Architecture Search based on Cartesian Genetic Programming Coding Method

Xuan Wu,Xiuyi Zhang,Linhan Jia,Liang Chen,Yanchun Liang,You Zhou,Chunguo Wu

2021-01-01

Abstract: Neural architecture search (NAS) is a hot topic in the field of AutoML, and has begun to outperform human-designed architectures on many machine learning tasks. Motivated by the natural representation form of neural networks by the Cartesian genetic programming (CGP), we propose an evolutionary approach of NAS based on CGP, called CPGNAS, for CNN architectures solving sentence classification task. To evolve the CNN architectures under the framework of CGP, the existing key operations are identified as the types of function nodes of CGP and the evolutionary operations are designed based on evolutionary strategy (ES). The experimental results show that the searched architecture can reach the accuracy of human-designed architectures. The ablation tests identify the Attention function as the single key function node and the Convolution and Attention as the joint key function nodes. However, the linear transformations along could keep the accuracy of evolved architectures over 70%, which is worth of investigating in the future.
KNAS: Green Neural Architecture Search

Jingjing Xu,Liang Zhao,Junyang Lin,Rundong Gao,Xu Sun,Hongxia Yang

DOI: https://doi.org/10.48550/arXiv.2111.13293

2021-11-26

Abstract:Many existing neural architecture search (NAS) solutions rely on downstream training for architecture evaluation, which takes enormous computations. Considering that these computations bring a large carbon footprint, this paper aims to explore a green (namely environmental-friendly) NAS solution that evaluates architectures without training. Intuitively, gradients, induced by the architecture itself, directly decide the convergence and generalization results. It motivates us to propose the gradient kernel hypothesis: Gradients can be used as a coarse-grained proxy of downstream training to evaluate random-initialized networks. To support the hypothesis, we conduct a theoretical analysis and find a practical gradient kernel that has good correlations with training loss and validation performance. According to this hypothesis, we propose a new kernel based architecture search approach KNAS. Experiments show that KNAS achieves competitive results with orders of magnitude faster than "train-then-test" paradigms on image classification tasks. Furthermore, the extremely low search cost enables its wide applications. The searched network also outperforms strong baseline RoBERTA-large on two text classification tasks. Codes are available at \url{<a class="link-external link-https" href="https://github.com/Jingjing-NLP/KNAS" rel="external noopener nofollow">this https URL</a>} .

Machine Learning
Guided evolutionary Neural Architecture Search with efficient performance estimation

Vasco Lopes,Miguel Santos,Bruno Degardin,Luís A. Alexandre

DOI: https://doi.org/10.1016/j.neucom.2024.127509

IF: 6

2024-03-17

Neurocomputing

Abstract:Neural Architecture Search (NAS) methods have been successfully applied to image tasks with excellent results. However, NAS methods are often complex and tend to converge to local minima as soon as generated architectures yield good results. This paper proposes GEA, a novel approach for guided NAS. GEA guides the evolution by exploring the search space by generating and evaluating several architectures in each generation at initialisation stage using a zero-proxy estimator, where only the highest-scoring architecture is trained and kept for the next generation. Subsequently, GEA continuously extracts knowledge about the search space without increased complexity by generating several off-springs from an existing architecture at each generation. Moreover, GEA forces exploitation of the most performant architectures by descendant generation while simultaneously driving exploration through parent mutation and favouring younger architectures to the detriment of older ones. Experimental results demonstrate the effectiveness of the proposed method, and extensive ablation studies evaluate the importance of different parameters. Results show that GEA achieves competitive results on all data sets of NAS-Bench-101, NAS-Bench-201 and TransNAS-Bench-101 benchmarks, as well as in the DARTS search space.

computer science, artificial intelligence
EGNAS: Efficient Graph Neural Architecture Search Through Evolutionary Algorithm

Younkyung Jwa,Chang Wook Ahn,Man-Je Kim

DOI: https://doi.org/10.3390/math12233828

IF: 2.4

2024-12-05

Mathematics

Abstract:The primary objective of our research is to enhance the efficiency and effectiveness of Neural Architecture Search (NAS) with regard to Graph Neural Networks (GNNs). GNNs have emerged as powerful tools for learning from unstructured network data, compensating for several known limitations of Convolutional Neural Networks (CNNs). However, the automatic search for optimal GNN architectures has seen little progressive advancement so far. To address this gap, we introduce the Efficient Graph Neural Architecture Search (EGNAS), a method that leverages the advantages of evolutionary search strategies. EGNAS incorporates inherited parameter sharing, allowing offspring to inherit parameters from their parents, and utilizes half epochs to improve optimization stability. In addition, EGNAS employs a combined evolutionary search, which explores both the model structure and the hyperparameters within a large search space, resulting in improved performance. Our experimental results demonstrate that EGNAS outperforms state-of-the-art methods in node classification tasks on the Cora, Citeseer, and PubMed datasets while maintaining a high degree of computational efficiency. In particular, EGNAS is the fastest GNN architecture search method in terms of search time, particularly when compared to precedently suggested evolutionary search strategies, delivering performance up to 40 times faster.

mathematics
Architecture search of accurate and lightweight CNNs using genetic algorithm

Jiayu Liang,Hanqi Cao,Yaxin Lu,Mingming Su

DOI: https://doi.org/10.1007/s10710-024-09484-4

2024-04-03

Genetic Programming and Evolvable Machines

Abstract:Convolutional neural networks (CNNs) are popularly-used in various AI fields, yet the design of CNN architectures heavily depends on domain expertise. Evolutionary neural architecture search (ENAS) methods can search for neural architectures automatically using evolutionary computation algorithms, e.g. genetic algorithm. However, most existing ENAS methods solely focus on the network accuracy, which leads to large-sized networks to be evolved and huge cost in computation resources and search time. Even though there are ENAS works using multi-objective techniques to optimize both the accuracy and size of CNNs, they are complex and time/resource-consuming. In this work, two new ENAS methods are designed, which aim to evolve both accurate and lightweight CNN architectures efficiently using genetic algorithm (GA). They are termed as GACNN_WS (GA CNN Weighted Sum) and GACNN_LE (GA CNN Local Elitism) respectively. Specifically, GACNN_WS designs a weighted-sum fitness of two items (i.e. accuracy and size) to evaluate candidate networks. GACNN_LE sets the accuracy as its fitness like most other ENAS methods, and designs a local elitism strategy to consider the network size. Thus, GACNN_WS and GACNN_LE can search for both accurate and lightweight CNNs without using multi-objective techniques. Results show that the proposed methods have better search ability than state-of-the-art NAS methods, which consume less time and generate better CNNs with lower error rates and parameter numbers for classification on CIFAR-10. Moreover, the evolved CNNs of the proposed methods generally perform better than eleven hand-designed CNNs.

computer science, artificial intelligence, theory & methods
Searching for A Robust Neural Architecture in Four GPU Hours

Xuanyi Dong,Yi Yang

DOI: https://doi.org/10.48550/arXiv.1910.04465

2019-10-10

Computer Vision and Pattern Recognition

Abstract:Conventional neural architecture search (NAS) approaches are based on reinforcement learning or evolutionary strategy, which take more than 3000 GPU hours to find a good model on CIFAR-10. We propose an efficient NAS approach learning to search by gradient descent. Our approach represents the search space as a directed acyclic graph (DAG). This DAG contains billions of sub-graphs, each of which indicates a kind of neural architecture. To avoid traversing all the possibilities of the sub-graphs, we develop a differentiable sampler over the DAG. This sampler is learnable and optimized by the validation loss after training the sampled architecture. In this way, our approach can be trained in an end-to-end fashion by gradient descent, named Gradient-based search using Differentiable Architecture Sampler (GDAS). In experiments, we can finish one searching procedure in four GPU hours on CIFAR-10, and the discovered model obtains a test error of 2.82\% with only 2.5M parameters, which is on par with the state-of-the-art. Code is publicly available on GitHub: https://github.com/D-X-Y/NAS-Projects.
Evolving graph convolutional networks for neural architecture search

George Kyriakides,Konstantinos Margaritis

DOI: https://doi.org/10.1007/s00521-021-05979-8

2021-04-15

Neural Computing and Applications

Abstract:As neural architecture search (NAS) becomes an increasingly adopted method to design network architectures, various methods have been proposed to speedup the process. Besides proxy evaluation tasks, weight sharing, and scaling down the evaluated architectures, performance-predicting models exhibit multiple advantages. Eliminating the need to train candidate architectures and enabling transfer learning between datasets, researchers can also utilize them as a surrogate function for Bayesian optimization. On the other hand, graph convolutional networks (GCNs) have also been increasingly adopted for various tasks, enabling deep learning techniques on graphs without feature engineering. In this paper, we employ an evolutionary-based NAS method to evolve GCNs for the problem of predicting the relative performance of various architectures included in the NAS-Bench-101 dataset. By fine-tuning the architecture generated by our methodology, we manage to achieve a Kendall's tau correlation coefficient of 0.907 between 1050 completely unseen architectures, utilizing only 450 samples, while also outperforming a strong baseline on the same task. Furthermore, we validate our method on custom global search space architectures, generated for the Fashion-MNIST dataset.

computer science, artificial intelligence
Reinforced Evolutionary Neural Architecture Search

Yukang Chen,Gaofeng Meng,Qian Zhang,Shiming Xiang,Chang Huang,Lisen Mu,Xinggang Wang

DOI: https://doi.org/10.48550/arXiv.1808.00193

2019-04-10

Abstract:Neural Architecture Search (NAS) is an important yet challenging task in network design due to its high computational consumption. To address this issue, we propose the Reinforced Evolutionary Neural Architecture Search (RE- NAS), which is an evolutionary method with the reinforced mutation for NAS. Our method integrates reinforced mutation into an evolution algorithm for neural architecture exploration, in which a mutation controller is introduced to learn the effects of slight modifications and make mutation actions. The reinforced mutation controller guides the model population to evolve efficiently. Furthermore, as child models can inherit parameters from their parents during evolution, our method requires very limited computational resources. In experiments, we conduct the proposed search method on CIFAR-10 and obtain a powerful network architecture, RENASNet. This architecture achieves a competitive result on CIFAR-10. The explored network architecture is transferable to ImageNet and achieves a new state-of-the-art accuracy, i.e., 75.7% top-1 accuracy with 5.36M parameters on mobile ImageNet. We further test its performance on semantic segmentation with DeepLabv3 on the PASCAL VOC. RENASNet outperforms MobileNet-v1, MobileNet-v2 and NASNet. It achieves 75.83% mIOU without being pre-trained on COCO.

Neural and Evolutionary Computing,Computer Vision and Pattern Recognition,Machine Learning
Genetic-GNN: Evolutionary architecture search for Graph Neural Networks

Min Shi,Yufei Tang,Xingquan Zhu,Yu Huang,David Wilson,Yuan Zhuang,Jianxun Liu

DOI: https://doi.org/10.1016/j.knosys.2022.108752

2022-04-01

Abstract:Neural architecture search (NAS) has seen significant attention throughout the computational intelligence research community and has pushed forward the state-of-the-art of many neural models to address grid-like data such as texts and images. However, little work has been done on Graph Neural Network (GNN) models dedicated to unstructured network data. Given the huge number of choices and combinations of components such as aggregators and activation functions, determining the suitable GNN model for a specific problem normally necessitates tremendous expert knowledge and laborious trials. In addition, the moderate change of hyperparameters such as the learning rate and dropout rate would dramatically impact the learning capacity of a GNN model. In this paper, we propose a novel framework through the evolution of individual models in a large GNN architecture searching space. Instead of simply optimizing the model structures, an alternating evolution process is performed between GNN model structures and hyperparameters to dynamically approach the optimal fit of each other. Experiments and validations demonstrate that evolutionary NAS is capable of matching existing state-of-the-art reinforcement learning methods for both transductive and inductive graph representation learning and node classification.

computer science, artificial intelligence
Neural Architecture Search Based on A Multi-objective Evolutionary Algorithm With Probability Stack

Yu Xue,Chen Chen,Adam Slowik

DOI: https://doi.org/10.1109/tevc.2023.3252612

IF: 16.497

2023-01-01

IEEE Transactions on Evolutionary Computation

Abstract:With the emergence of deep neural networks, many research fields, such as image classification, object detection, speech recognition, natural language processing, machine translation and automatic driving, have made major breakthroughs in technology and the research achievements have been successfully applied in many real-life applications. Combining evolutionary computation and neural architecture search (NAS) is an important approach to improve the performance of deep neural networks. Usually, the related researchers only focus on precision. Thus, the searched neural architectures always perform poorly in the other indexes such as time cost. In this paper, a multi-objective evolutionary algorithm with a probability stack (MOEA-PS) is proposed for NAS, which considers the two objects of precision and time consumption. MOEA-PS uses an adjacency list to represent the internal structure of deep neural networks. Besides, a unique mechanism is introduced into the multi-objective genetic algorithm to guide the process of crossover and mutation when generating offspring. Furthermore, the structure blocks are stacked using a proxy model to generate deep neural networks. The results of the experiments on Cifar-10 and Cifar-100 demonstrate that the proposed algorithm has a similar error rate compared with the most advanced NAS algorithms, but the time cost is lower. Finally, the network structure searched on Cifar-10 is transferred directly to the ImageNet dataset, which can achieve 73.6% classification accuracy.

computer science, artificial intelligence, theory & methods
GPT-NAS: Evolutionary Neural Architecture Search with the Generative Pre-Trained Model

Caiyang Yu,Xianggen Liu,Yifan Wang,Yun Liu,Wentao Feng,Xiong Deng,Chenwei Tang,Jiancheng Lv

DOI: https://doi.org/10.26599/BDMA.2024.9020036

2024-10-29

Abstract:Neural Architecture Search (NAS) has emerged as one of the effective methods to design the optimal neural network architecture automatically. Although neural architectures have achieved human-level performances in several tasks, few of them are obtained from the NAS method. The main reason is the huge search space of neural architectures, making NAS algorithms inefficient. This work presents a novel architecture search algorithm, called GPT-NAS, that optimizes neural architectures by Generative Pre-Trained (GPT) model with an evolutionary algorithm (EA) as the search strategy. In GPT-NAS, we assume that a generative model pre-trained on a large-scale corpus could learn the fundamental law of building neural architectures. Therefore, GPT-NAS leverages the GPT model to propose reasonable architecture components given the basic one and then utilizes EAs to search for the optimal solution. Such an approach can largely reduce the search space by introducing prior knowledge in the search process. Extensive experimental results show that our GPT-NAS method significantly outperforms seven manually designed neural architectures and thirteen architectures provided by competing NAS methods. In addition, our experiments also indicate that the proposed algorithm improves the performance of finely tuned neural architectures by up to about 12% compared to those without GPT, further demonstrating its effectiveness in searching neural architectures.

Computer Vision and Pattern Recognition,Artificial Intelligence
Improved Search in Neuroevolution Using a Neural Architecture Classifier With the CNN Architecture Encoding as Feature Vector

Jhon I. Pilataxi,Jorge E. Zambrano,Claudio A. Perez,Kevin W. Bowyer

DOI: https://doi.org/10.1109/access.2024.3355804

IF: 3.9

2024-01-01

IEEE Access

Abstract:Designing Convolutional Neural Networks (CNNs) for a specific task requires not only Deep Learning expertise but also knowledge of the problem. The goal of Neuroevolution is to find CNN architectures automatically through evolution. The search time, however, is a critical problem in Neuroevolution since multiple CNNs must be trained in the evolutionary process. In this work, we propose a Neural Architecture Classifier (NAC) to avoid training architectures that would not have good performance, based on knowledge of previously trained architectures. The NAC evaluates each CNN using the CNN architecture encoding as the input. A genetic algorithm (GA) is used for evolution, and a search space with few restrictions; hence, CNN architectures can have any width, depth or shape as well as the suitable number of skip connections. We applied this methodology to solving pattern recognition problems on six well-known datasets: five subsets of the MNIST-Variations (MNIST-V) dataset and the CIFAR-10 dataset. Experiments demonstrated that integrating the NAC in the GA reduces the search time by up to 44% compared to conventional GAs. Additionally, the architectures found by evolution with our classifier achieved a better performance (4.6% on average) than those found by traditional evolution. Our results improved the state-of-the-art by 5.2% (0.33% error reduction) on the most difficult dataset of the MNIST-V (MRDBI), and an average of 4% (0.24% error reduction) on the MNIST-V datasets. Additionally, our results on CIFAR-10 are close to those of the state-of-the-art but with significantly reduced search time. The application of a classifier can be relevant and useful not only for genetic algorithms, but also for other evolutionary algorithms.

computer science, information systems,telecommunications,engineering, electrical & electronic
Generative Adversarial Neural Architecture Search

Seyed Saeed Changiz Rezaei,Fred X. Han,Di Niu,Mohammad Salameh,Keith Mills,Shuo Lian,Wei Lu,Shangling Jui

DOI: https://doi.org/10.48550/arXiv.2105.09356

2021-06-23

Abstract:Despite the empirical success of neural architecture search (NAS) in deep learning applications, the optimality, reproducibility and cost of NAS schemes remain hard to assess. In this paper, we propose Generative Adversarial NAS (GA-NAS) with theoretically provable convergence guarantees, promoting stability and reproducibility in neural architecture search. Inspired by importance sampling, GA-NAS iteratively fits a generator to previously discovered top architectures, thus increasingly focusing on important parts of a large search space. Furthermore, we propose an efficient adversarial learning approach, where the generator is trained by reinforcement learning based on rewards provided by a discriminator, thus being able to explore the search space without evaluating a large number of architectures. Extensive experiments show that GA-NAS beats the best published results under several cases on three public NAS benchmarks. In the meantime, GA-NAS can handle ad-hoc search constraints and search spaces. We show that GA-NAS can be used to improve already optimized baselines found by other NAS methods, including EfficientNet and ProxylessNAS, in terms of ImageNet accuracy or the number of parameters, in their original search space.

Machine Learning,Computer Vision and Pattern Recognition
Optimizing Neural Architecture Search using Limited GPU Time in a Dynamic Search Space: A Gene Expression Programming Approach

Jeovane Honorio Alves,Lucas Ferrari de Oliveira

DOI: https://doi.org/10.1109/CEC48606.2020.9185856

2020-05-16

Abstract:Efficient identification of people and objects, segmentation of regions of interest and extraction of relevant data in images, texts, audios and videos are evolving considerably in these past years, which deep learning methods, combined with recent improvements in computational resources, contributed greatly for this achievement. Although its outstanding potential, development of efficient architectures and modules requires expert knowledge and amount of resource time available. In this paper, we propose an evolutionary-based neural architecture search approach for efficient discovery of convolutional models in a dynamic search space, within only 24 GPU hours. With its efficient search environment and phenotype representation, Gene Expression Programming is adapted for network's cell generation. Despite having limited GPU resource time and broad search space, our proposal achieved similar state-of-the-art to manually-designed convolutional networks and also NAS-generated ones, even beating similar constrained evolutionary-based NAS works. The best cells in different runs achieved stable results, with a mean error of 2.82% in CIFAR-10 dataset (which the best model achieved an error of 2.67%) and 18.83% for CIFAR-100 (best model with 18.16%). For ImageNet in the mobile setting, our best model achieved top-1 and top-5 errors of 29.51% and 10.37%, respectively. Although evolutionary-based NAS works were reported to require a considerable amount of GPU time for architecture search, our approach obtained promising results in little time, encouraging further experiments in evolutionary-based NAS, for search and network representation improvements.

Computer Vision and Pattern Recognition,Machine Learning,Neural and Evolutionary Computing
APENAS: An Asynchronous Parallel Evolution Based Multi-objective Neural Architecture Search

Mengtao Hu,Li Liu,Wei Wang,Yao Liu

DOI: https://doi.org/10.1109/ispa-bdcloud-socialcom-sustaincom51426.2020.00045

2020-12-01

Abstract:Machine learning is widely used in pattern classification, image processing and speech recognition. Neural architecture search (NAS) could reduce the dependence of human experts on machine learning effectively. Due to the high complexity of NAS, the tradeoff between time consumption and classification accuracy is vital. This paper presents APENAS, an asynchronous parallel evolution based multi-objective neural architecture search, using the classification accuracy and the number of parameters as objectives, encoding the network architectures as individuals. To make full use of computing resource, we propose a multi-generation undifferentiated fusion scheme to achieve asynchronous parallel evolution on multiple GPUs or CPUs, which speeds up the process of NAS. Accordingly, we propose an election pool and a buffer pool for two-layer filtration of individuals. The individuals are sorted in the election pool by non-dominated sorting and filtered in the buffer pool by the roulette algorithm to improve the elitism of the Pareto front. APENAS is evaluated on the CIFAR-10 and CIFAR-100 datasets [25]. The experimental results demonstrate that APENAS achieves 90.05% accuracy on CIFAR-10 with only 0.07 million parameters, which is comparable to state of the art. Especially, APENAS has high parallel scalability, achieving 92.5% parallel efficiency on 64 nodes.
Genetic Network Architecture Search

Hai Victor Habi,Gil Rafalovich

DOI: https://doi.org/10.48550/arXiv.1907.02871

2019-07-05

Neural and Evolutionary Computing

Abstract:We propose a method for learning the neural network architecture that based on Genetic Algorithm (GA). Our approach uses a genetic algorithm integrated with standard Stochastic Gradient Descent(SGD) which allows the sharing of weights across all architecture solutions. The method uses GA to design a sub-graph of Convolution cell which maximizes the accuracy on the validation-set. Through experiments, we demonstrate this methods performance on both CIFAR10 and CIFAR100 dataset with an accuracy of 96% and 80.1%. The code and result of this work available in GitHub:https://github.com/haihabi/GeneticNAS.
Evolutionary Neural Architecture Search for Image Restoration

Gerard Jacques van Wyk,Anna Sergeevna Bosman

DOI: https://doi.org/10.48550/arXiv.1812.05866

2019-03-30

Abstract:Convolutional neural network (CNN) architectures have traditionally been explored by human experts in a manual search process that is time-consuming and ineffectively explores the massive space of potential solutions. Neural architecture search (NAS) methods automatically search the space of neural network hyperparameters in order to find optimal task-specific architectures. NAS methods have discovered CNN architectures that achieve state-of-the-art performance in image classification among other tasks, however the application of NAS to image-to-image regression problems such as image restoration is sparse. This paper proposes a NAS method that performs computationally efficient evolutionary search of a minimally constrained network architecture search space. The performance of architectures discovered by the proposed method is evaluated on a variety of image restoration tasks applied to the ImageNet64x64 dataset, and compared with human-engineered CNN architectures. The best neural architectures discovered using only 2 GPU-hours of evolutionary search exhibit comparable performance to the human-engineered baseline architecture.

Neural and Evolutionary Computing

A 32-year-old man with recurrent cholangitis

Enhanced Gradient for Differentiable Architecture Search

G-EvoNAS: Evolutionary Neural Architecture Search Based on Network Growth

Neural Architecture Search with Progressive Evaluation and Sub-Population Preservation

Neural Architecture Search based on Cartesian Genetic Programming Coding Method

KNAS: Green Neural Architecture Search

Guided evolutionary Neural Architecture Search with efficient performance estimation

EGNAS: Efficient Graph Neural Architecture Search Through Evolutionary Algorithm

Architecture search of accurate and lightweight CNNs using genetic algorithm

Searching for A Robust Neural Architecture in Four GPU Hours

Evolving graph convolutional networks for neural architecture search

Reinforced Evolutionary Neural Architecture Search

Genetic-GNN: Evolutionary architecture search for Graph Neural Networks

Neural Architecture Search Based on A Multi-objective Evolutionary Algorithm With Probability Stack

GPT-NAS: Evolutionary Neural Architecture Search with the Generative Pre-Trained Model

Improved Search in Neuroevolution Using a Neural Architecture Classifier With the CNN Architecture Encoding as Feature Vector

Generative Adversarial Neural Architecture Search

Optimizing Neural Architecture Search using Limited GPU Time in a Dynamic Search Space: A Gene Expression Programming Approach

APENAS: An Asynchronous Parallel Evolution Based Multi-objective Neural Architecture Search

Genetic Network Architecture Search

Evolutionary Neural Architecture Search for Image Restoration