Abstract:Neural network architecture search automatically configures a set of network architectures according to the targeted rules. Thus, it relieves the human-dependent effort and repetitive resources consumption for designing neural network architectures and makes the task of finding the optimum network architecture with better performance much more accessible. Network architecture search methods based on differentiable architecture search (DARTS), however, introduces parameter redundancy. To address this issue, this work presents a novel method for optimizing network architectures that combines DARTS with generative adversarial learning (GAL). We first find the module structures utilizing the DARTS algorithm. Afterwards, the retrieved modules are stacked to derive the initial neural network architecture. Next, the GAL is used to prune some branches of the initial neural network, thereby obtaining the final neural network architecture. The proposed DARTS-GAL method re-optimizes the network architecture searched by DARTS to simplify the network connection and reduce network parameters without compromising network performance. Experimental results on benchmark datasets, i.e., Mixed National Institute of Standards and Technology (MNIST), FashionMNIST, Canadian Institute for Advanced Research10 (CIFAR10), Canadian Institute for Advanced Research100 (CIAFR100), Cats vs Dogs, and voiceprint recognition datasets, indicate that the test accuracies of the DARTS-GAL are higher than those of the DARTS in the majority of the cases. In particular, the proposed solution exhibits an improvement in accuracy by 7.35% on CIFAR10 compared with DARTS, attaining the state-of-the-art result of 99.60%. Additionally, the number of network parameters derived by the DARTS-GAL is significantly lower than that by the DARTS method, with a pruning rate of 62.3% at the highest case.

RL-DARTS: Differentiable neural architecture search via reinforcement-learning-based meta-optimizer

Making Differentiable Architecture Search less local

$\alpha$ DARTS Once More: Enhancing Differentiable Architecture Search by Masked Image Modeling

PO-DARTS: Post-optimizing the Architectures Searched by Differentiable Architecture Search Algorithms

A neural network architecture optimizer based on DARTS and generative adversarial learning

iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients

Cyclic Differentiable Architecture Search

A new differentiable architecture search method for optimizing convolutional neural networks in the digital twin of intelligent robotic grasping

PD-DARTS - Progressive Discretization Differentiable Architecture Search.

Noisy Differentiable Architecture Search

Improving Differentiable Architecture Search via self-distillation

Efficient Search for Efficient Architecture.

Inner Loop-Based Modified Differentiable Architecture Search

D-DARTS: Distributed Differentiable Architecture Search

Operation-level Progressive Differentiable Architecture Search

Differentiable Neural Architecture Search Via Proximal Iterations.

OStr-DARTS: Differentiable Neural Architecture Search based on Operation Strength

IDARTS - Interactive Differentiable Architecture Search.

Enhanced Differentiable Architecture Search Based on Asymptotic Regularization

Differentiable Neural Architecture Search with Morphism-based Transformable Backbone Architectures

Single-DARTS: Towards Stable Architecture Search