Abstract:Neural Architecture Search (NAS), aiming to automatically search for neural structure that performs the best, has attracted lots of attentions from both the academy and industry. However, most existing works assume each layer accepts a fixed number of inputs from previous layers, ignoring the flexibility of receiving inputs from an arbitrary number of previous layers. Allowing to receive inputs from an arbitrary number of layers benefits in introducing far more possible combinations of connections among layers, which may also result in much more complex structural relations in architectures. Existing works fail to capture structural correlations among different layers, thus limiting the ability to discover the optimal architecture. To overcome the weakness of existing methods, we study the NAS problem by assuming an arbitrary number of inputs for each layer and capturing the structural correlations among different layers in this paper. Nevertheless, besides the complex structural correlations, considering an arbitrary number of inputs for each layer may also lead to a fully connected structure with up to O(n2) connections for n layers, posing great challenges to efficiently handle polynomial numbers of connections among different layers. To tackle this challenge, we propose a Graph Q Network for NAS (GQNAS), where the states and actions are redefined for searching architectures with input from an arbitrary number of layers. Concretely, we regard a neural architecture as a directed acyclic graph and use graph neural network (GNN) as the Q-function approximation in deep Q network (DQN) to capture the complex structural relations between different layers for obtaining accurate Q-values. Our extensive experiments show that the proposed GQNAS model is able to achieve better performances than several state-of-the-art approaches.

GQNAS - Graph Q Network for Neural Architecture Search.

Simplifying Architecture Search for Graph Neural Network

Auto-GNN: Neural Architecture Search of Graph Neural Networks

Graph Differentiable Architecture Search with Structure Learning

Rethinking Graph Neural Architecture Search From Message-Passing.

DFG-NAS: Deep and Flexible Graph Neural Architecture Search

GraphNAS++: Distributed Architecture Search for Graph Neural Networks

Rethinking Graph Neural Network Search from Message-passing

NAS-Bench-Graph: Benchmarking Graph Neural Architecture Search

Latency-aware Neural Architecture Performance Predictor with Query-to-Tier Technique

GPT-NAS: Evolutionary Neural Architecture Search with the Generative Pre-Trained Model

Genetic-GNN: Evolutionary architecture search for Graph Neural Networks

Learned Low Precision Graph Neural Networks

Posterior-Guided Neural Architecture Search

Graph HyperNetworks for Neural Architecture Search

A Generic Graph-Based Neural Architecture Encoding Scheme with Multifaceted Information

Probabilistic Dual Network Architecture Search on Graphs

Search for Deep Graph Neural Networks

Knowledge-aware Evolutionary Graph Neural Architecture Search

GraphPNAS: Learning Distribution of Good Neural Architectures via Deep Graph Generative Models

Manas: multi-agent neural architecture search