NeuE: Automated Neural Network Ensembles for Edge Intelligence

Yang Bai,Lixing Chen,Jie Xu
DOI: https://doi.org/10.1109/tetc.2022.3214931
2022-01-01
IEEE Transactions on Emerging Topics in Computing
Abstract:Artificial Intelligence (AI) applications have been established in the mobile industry and are decisively determining the progress in entrepreneurial value creation. This article explores the potential of Edge Computing to enhance the performance of AI applications. In particular, a DNN ensemble formation (DEF) problem is studied which judiciously recruits members for DNN ensembles considering the device heterogeneity, computing resource limitation, and service deadline of edge computing systems, in an attempt to optimize the performance of edge AI services. We design a novel algorithm called Neural Ensemble (NeuE) to solve the DEF problem. NeuE involves an online learning process that learns the in-practice performance of DNN ensembles and adaptively forms DNN ensembles according to the features of admitted tasks. It leverages the framework of contextual multi-armed bandit and follows the constraints of computing resource limitation and service deadline. We also show theoretically that NeuE provides asymptotic optimality. However, NeuE suffers from poor scalability due to exponentially-growing ensemble decision space. We then propose a variant of NeuE, called NeuE-S, to expedite NeuE. NeuE-S identifies representative ensemble decisions using similarities of ensemble decisions and carries out learning with a reduced decision space. We show via theoretical analysis that NeuE-S drastically reduces the computation complexity with negligible performance loss. We implement our method on an edge computing testbed. The results show that our method dramatically improves the performance of edge AI services.
computer science, information systems,telecommunications
What problem does this paper attempt to address?