Abstract:Jointly learning from multiple datasets can help building versatile intelligent systems yet may give rise to serious concerns of data privacy and model selection. Specifically, on the one hand, these datasets can be distributed at various local clients, who may not be willing or do not ought to share data with each other. On the other hand, it is unrealistic to choose a model architecture that can well suit the disparate patterns and distributions carried by the various datasets in a priori. Whereas many works in federated learning [1] and neural architecture search [2] have been proposed to address one of the two concerns, very few have attempted the both. To close the gap, in this paper we deliver a framework, termed Multi-Granular Federated Neural Architecture Search (MGFNAS), to enable the automation of model architecture search in a federated and thus privacy-preserved setting. We argue that our MGFNAS framework is general in the sense that it does not impose any restriction on the search space or strategy, such that most existing neural architecture search techniques can be readily implemented in. The main idea of our framework is to search the optimal neural network architecture in two levels of granularity, enabling the neural-operator-based micro-level search and the cell-based macro-level search. The main challenge of implementing our framework lies in the fact that, due to the decentralized nature, the local architectures searched by multiple clients can differ drastically in order to fit their own datasets, while a general method to form the global model by aggregating the local architectures in both micro and macro levels is missing. To solve the issue, we propose a novel aggregation function, named Network Architecture Probabilistic Aggregation (NAPA). The key idea of our NAPA function is to treat the network architectures as graphs, of which the sub-graph structures being frequently appeared across multiple clients are modeled by probabilistic distributions. At each round, a global model is formed by sampling from those distributions in an exploration-exploitation fashion. Extensive experiments are carried out, and the results substantiate the viability and effectiveness of our proposed framework.

Federated Bayesian Optimization for Privacy-Preserving Neural Architecture Search

Neural Architecture Search over Decentralized Data.

Resource-Aware Federated Neural Architecture Search over Heterogeneous Mobile Devices

Privacy-Preserving Federated Neural Architecture Search with Enhanced Robustness for Edge Computing

Privacy-Preserving Multi-Granular Federated Neural Architecture Search - A General Framework.

Differentially-private Federated Neural Architecture Search

BayesNAS: A Bayesian Approach for Neural Architecture Search

Multi-shot NAS for Discovering Adversarially Robust Convolutional Neural Architectures at Targeted Capacities

Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Search

SuperFedNAS: Cost-Efficient Federated Neural Architecture Search for On-Device Inference

Self-supervised Cross-silo Federated Neural Architecture Search

Finch: Enhancing Federated Learning with Hierarchical Neural Architecture Search

Federated Neural Architecture Search for Medical Data Security

Deeply Supervised Block-Wise Neural Architecture Search

Towards a configurable and non-hierarchical search space for NAS

NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension

From federated learning to federated neural architecture search: a survey

Real-time Federated Evolutionary Neural Architecture Search

FAQS: Communication-efficient Federate DNN Architecture and Quantization Co-Search for personalized Hardware-aware Preferences

Bi-fidelity Evolutionary Multiobjective Search for Adversarially Robust Deep Neural Architectures

Data Aware Neural Architecture Search