Abstract:Jointly learning from multiple datasets can help building versatile intelligent systems yet may give rise to serious concerns of data privacy and model selection. Specifically, on the one hand, these datasets can be distributed at various local clients, who may not be willing or do not ought to share data with each other. On the other hand, it is unrealistic to choose a model architecture that can well suit the disparate patterns and distributions carried by the various datasets in a priori. Whereas many works in federated learning [1] and neural architecture search [2] have been proposed to address one of the two concerns, very few have attempted the both. To close the gap, in this paper we deliver a framework, termed Multi-Granular Federated Neural Architecture Search (MGFNAS), to enable the automation of model architecture search in a federated and thus privacy-preserved setting. We argue that our MGFNAS framework is general in the sense that it does not impose any restriction on the search space or strategy, such that most existing neural architecture search techniques can be readily implemented in. The main idea of our framework is to search the optimal neural network architecture in two levels of granularity, enabling the neural-operator-based micro-level search and the cell-based macro-level search. The main challenge of implementing our framework lies in the fact that, due to the decentralized nature, the local architectures searched by multiple clients can differ drastically in order to fit their own datasets, while a general method to form the global model by aggregating the local architectures in both micro and macro levels is missing. To solve the issue, we propose a novel aggregation function, named Network Architecture Probabilistic Aggregation (NAPA). The key idea of our NAPA function is to treat the network architectures as graphs, of which the sub-graph structures being frequently appeared across multiple clients are modeled by probabilistic distributions. At each round, a global model is formed by sampling from those distributions in an exploration-exploitation fashion. Extensive experiments are carried out, and the results substantiate the viability and effectiveness of our proposed framework.

Differentially-private Federated Neural Architecture Search

DPNAS: Neural Architecture Search for Deep Learning with Differential Privacy

Federated Bayesian Optimization for Privacy-Preserving Neural Architecture Search

Privacy-Preserving Federated Neural Architecture Search with Enhanced Robustness for Edge Computing

Neural Architecture Search over Decentralized Data.

Privacy-Preserving Multi-Granular Federated Neural Architecture Search - A General Framework.

From federated learning to federated neural architecture search: a survey

Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Search

EFNAS: Efficient Federated Neural Architecture Search Across AIoT Devices

Collaborative Neural Architecture Search for Personalized Federated Learning

Resource-Aware Federated Neural Architecture Search over Heterogeneous Mobile Devices

Self-supervised Cross-silo Federated Neural Architecture Search

DP-FL: a Novel Differentially Private Federated Learning Framework for the Unbalanced Data

Differentially Private Federated Learning with an Adaptive Noise Mechanism

Real-time Federated Evolutionary Neural Architecture Search

Discretization-aware architecture search

DFairNAS: A Dataflow Fairness Approach to Training NAS Neural Networks.

DATA: Differentiable ArchiTecture Approximation with Distribution Guided Sampling

Neighborhood-Aware Neural Architecture Search

DDPNAS: Efficient Neural Architecture Search Via Dynamic Distribution Pruning

SPIDER: Searching Personalized Neural Architecture for Federated Learning