Abstract:ProKube is a proactive container scaling and migration orchestrator to dynamically adjust the resources and container locations with a fair balance between cost and delay for ML inferences in the inherent heterogeneous, resource‐constrained, and distributed edge environments. Deep neural network (DNN) and machine learning (ML) models/ inferences produce highly accurate results demanding enormous computational resources. The limited capacity of end‐user smart gadgets drives companies to exploit computational resources in an edge‐to‐cloud continuum and host applications at user‐facing locations with users requiring fast responses. Kubernetes hosted inferences with poor resource request estimation results in service level agreement (SLA) violation in terms of latency and below par performance with higher end‐to‐end (E2E) delays. Lifetime static resource provisioning either hurts user experience for under‐resource provisioning or incurs cost with over‐provisioning. Dynamic scaling offers to remedy delay by upscaling leading to additional cost whereas a simple migration to another location offering latency in SLA bounds can reduce delay and minimize cost. To address this cost and delay challenges for ML inferences in the inherent heterogeneous, resource‐constrained, and distributed edge environment, we propose ProKube, which is a proactive container scaling and migration orchestrator to dynamically adjust the resources and container locations with a fair balance between cost and delay. ProKube is developed in conjunction with Google Kubernetes Engine (GKE) enabling cross‐cluster migration and/ or dynamic scaling. It further supports the regular addition of freshly collected logs into scheduling decisions to handle unpredictable network behavior. Experiments conducted in heterogeneous edge settings show the efficacy of ProKube to its counterparts cost greedy (CG), latency greedy (LG), and GeKube (GK). ProKube offers 68%, 7%, and 64% SLA violation reduction to CG, LG, and GK, respectively, and it improves cost by 4.77 cores to LG and offers more cost of 3.94 to CG and GK.

KOLE: breaking the scalability barrier for managing far edge nodes in cloud

Enhancing Edge Environment Scalability: Leveraging Kubernetes for Container Orchestration and Optimization

Extend Cloud to Edge with KubeEdge

KBMP: Kubernetes-Orchestrated IoT Online Battery Monitoring Platform

A Kubernetes-Oriented Edge Network Orchestrator for Heterogeneous Environment

Ultra-Reliable and Low-Latency Computing in the Edge with Kubernetes

KheOps: Cost-effective Repeatability, Reproducibility, and Replicability of Edge-to-Cloud Experiments

Achelous: Enabling Programmability, Elasticity, and Reliability in Hyperscale Cloud Networks.

Towards Seamless Serverless Computing Across an Edge-Cloud Continuum

EdgeOptimizer : A programmable containerized scheduler of time-critical tasks in Kubernetes -based edge-cloud clusters

CoopEdge: Cost-effective Server Deployment for Cooperative Multi-Access Edge Computing.

ProKube: Proactive Kubernetes Orchestrator for Inference in Heterogeneous Edge Computing

KCES: A Workflow Containerization Scheduling Scheme Under Cloud-Edge Collaboration Framework

An Open-Source Experimentation Framework for the Edge Cloud Continuum

Engineering and Experimentally Benchmarking a Serverless Edge Computing System

Kubernetes distributions for the edge: serverless performance evaluation

A Multi-Layered Distributed Computing Framework for Enhanced Edge Computing

CoopEdge: A Decentralized Blockchain-based Platform for Cooperative Edge Computing

BREAK: A Holistic Approach for Efficient Container Deployment among Edge Clouds

Komet: A Serverless Platform for Low-Earth Orbit Edge Services

Bené: On Demand Cost-Effective Scaling at the Edge