Abstract:Jointly learning from multiple datasets can help building versatile intelligent systems yet may give rise to serious concerns of data privacy and model selection. Specifically, on the one hand, these datasets can be distributed at various local clients, who may not be willing or do not ought to share data with each other. On the other hand, it is unrealistic to choose a model architecture that can well suit the disparate patterns and distributions carried by the various datasets in a priori. Whereas many works in federated learning [1] and neural architecture search [2] have been proposed to address one of the two concerns, very few have attempted the both. To close the gap, in this paper we deliver a framework, termed Multi-Granular Federated Neural Architecture Search (MGFNAS), to enable the automation of model architecture search in a federated and thus privacy-preserved setting. We argue that our MGFNAS framework is general in the sense that it does not impose any restriction on the search space or strategy, such that most existing neural architecture search techniques can be readily implemented in. The main idea of our framework is to search the optimal neural network architecture in two levels of granularity, enabling the neural-operator-based micro-level search and the cell-based macro-level search. The main challenge of implementing our framework lies in the fact that, due to the decentralized nature, the local architectures searched by multiple clients can differ drastically in order to fit their own datasets, while a general method to form the global model by aggregating the local architectures in both micro and macro levels is missing. To solve the issue, we propose a novel aggregation function, named Network Architecture Probabilistic Aggregation (NAPA). The key idea of our NAPA function is to treat the network architectures as graphs, of which the sub-graph structures being frequently appeared across multiple clients are modeled by probabilistic distributions. At each round, a global model is formed by sampling from those distributions in an exploration-exploitation fashion. Extensive experiments are carried out, and the results substantiate the viability and effectiveness of our proposed framework.

Privacy-Preserving Multi-Granular Federated Neural Architecture Search - A General Framework.

Neural Architecture Search over Decentralized Data.

From federated learning to federated neural architecture search: a survey

Collaborative Neural Architecture Search for Personalized Federated Learning

EFNAS: Efficient Federated Neural Architecture Search Across AIoT Devices

Federated Bayesian Optimization for Privacy-Preserving Neural Architecture Search

Resource-Aware Federated Neural Architecture Search over Heterogeneous Mobile Devices

Differentially-private Federated Neural Architecture Search

Self-supervised Cross-silo Federated Neural Architecture Search

Multi-Relational Graph Neural Architecture Search with Fine-grained Message Passing

MGAS: Multi-Granularity Architecture Search for Trade-Off Between Model Effectiveness and Efficiency

Real-time Federated Evolutionary Neural Architecture Search

Automated Graph Neural Network Search under Federated Learning Framework

FedAutoMRI: Federated Neural Architecture Search for MR Image Reconstruction

Federated Neural Architecture Search for Medical Data Security

Finch: Enhancing Federated Learning with Hierarchical Neural Architecture Search

FL-AGCNS: Federated Learning Framework for Automatic Graph Convolutional Network Search

SPIDER: Searching Personalized Neural Architecture for Federated Learning

A Multiscale Neural Architecture Search Framework for Multimodal Fusion

GNAS: A Greedy Neural Architecture Search Method for Multi-Attribute Learning

Pareto-Frontier-aware Neural Architecture Search