Abstract:We consider a resource-constrained Edge Device (ED), such as an IoT sensor or a microcontroller unit, embedded with a small-size ML model (S-ML) for a generic classification application and an Edge Server (ES) that hosts a large-size ML model (L-ML). Since the inference accuracy of S-ML is lower than that of the L-ML, offloading all the data samples to the ES results in high inference accuracy, but it defeats the purpose of embedding S-ML on the ED and deprives the benefits of reduced latency, bandwidth savings, and energy efficiency of doing local inference. In order to get the best out of both worlds, i.e., the benefits of doing inference on the ED and the benefits of doing inference on ES, we explore the idea of Hierarchical Inference (HI), wherein S-ML inference is only accepted when it is correct, otherwise the data sample is offloaded for L-ML inference. However, the ideal implementation of HI is infeasible as the correctness of the S-ML inference is not known to the ED. We propose an online meta-learning framework that the ED can use to predict the correctness of the S-ML inference. In particular, we propose to use the maximum softmax value output by S-ML for a data sample and decide whether to offload it or not. The resulting online learning problem turns out to be a Prediction with Expert Advice (PEA) problem with continuous expert space. We propose two different algorithms and prove sublinear regret bounds for them without any assumption on the smoothness of the loss function. We evaluate and benchmark the performance of the proposed algorithms for image classification application using four datasets, namely, Imagenette and Imagewoof, MNIST, and CIFAR-10.

XLDA: Linear Discriminant Analysis for Scaling Continual Learning to Extreme Classification at the Edge

Online Discriminative Dictionary Learning for Image Classification Based on Block-Coordinate Descent Method

Extreme Learning Machine Based Transfer Learning for Data Classification.

A Distributed Privacy-Preserving Framework for Deep Learning with Edge-Cloud Computing.

Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis

Edge-Cloud Collaborative Streaming Video Analytics with Multi-agent Deep Reinforcement Learning

Online Learning in Neural Decoding Using Incremental Linear Discriminant Analysis

Deep Encoders with Auxiliary Parameters for Extreme Classification.

A scalable supervised algorithm for dimensionality reduction on streaming data

Domain-Adaptive Online Active Learning for Real-Time Intelligent Video Analytics on Edge Devices

Accelerating Extreme Classification via Adaptive Feature Agglomeration

Least squares online linear discriminant analysis

Least Square Incremental Linear Discriminant Analysis

Regularized Deep Linear Discriminant Analysis

BigDL 2.0: Seamless Scaling of AI Pipelines from Laptops to Distributed Cluster

Deep Streaming Label Learning

Online Algorithms for Hierarchical Inference in Deep Learning applications at the Edge

An Incremental Linear Discriminant Analysis Using Fixed Point Method

DELTA: Decoupling Long-Tailed Online Continual Learning

DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learning

STREAMLINE: Streaming Active Learning for Realistic Multi-Distributional Settings