Abstract:Detection-driven real-time video analytics require continuous detection of objects contained in the video frames using deep learning models like YOLOV3, EfficientDet. However, running these detectors on each and every frame in resource-constrained edge devices is computationally intensive. By taking the temporal correlation between consecutive video frames into account, we note that detection outputs tend to be overlapping in successive frames. Elimination of similar consecutive frames will lead to a negligible drop in performance while offering significant performance benefits by reducing overall computation and communication costs. The key technical questions are, therefore, (a) how to identify which frames to be processed by the object detector, and (b) how many successive frames can be skipped (called skip-length) once a frame is selected to be processed. The overall goal of the process is to keep the error due to skipping frames as small as possible. We introduce a novel error vs processing rate optimization problem with respect to the object detection task that balances between the error rate and the fraction of frames filtering. Subsequently, we propose an off-line Reinforcement Learning (RL)-based algorithm to determine these skip-lengths as a state-action policy of the RL agent from a recorded video and then deploy the agent online for live video streams. To this end, we develop FrameHopper, an edge-cloud collaborative video analytics framework, that runs a lightweight trained RL agent on the camera and passes filtered frames to the server where the object detection model runs for a set of applications. We have tested our approach on a number of live videos captured from real-life scenarios and show that FrameHopper processes only a handful of frames but produces detection results closer to the oracle solution and outperforms recent state-of-the-art solutions in most cases.

Frame Offloading Scheduling Algorithm for Real-time Object Detection in Edge Environment

Characterization of Real-Time Object Detection Workloads on Vehicular Edge

AyE-Edge: Automated Deployment Space Search Empowering Accuracy Yet Efficient Real-Time Object Detection on the Edge

Optimizing Edge Offloading Decisions for Object Detection

OffloaD: Detection Failure-based Scheduler for Offloading Object Detection

Edge-assisted Online On-device Object Detection for Real-time Video Analytics.

A cooperative image object recognition framework and task offloading optimization in edge computing

TOD: Transprecise Object Detection to Maximise Real-Time Accuracy on the Edge

EdgeDuet: Tiling Small Object Detection for Edge Assisted Autonomous Mobile Vision.

Edge YOLO: Real-Time Intelligent Object Detection System Based on Edge-Cloud Cooperation in Autonomous Vehicles

Edge Computing Enabled Real-Time Video Analysis via Adaptive Spatial-Temporal Semantic Filtering

Accelerating real‐time object detection in high‐resolution video surveillance

Object Detection Acceleration Method by Improving Execution Efficiency of Edge AI Device

Edge Computing Driven Low-Light Image Dynamic Enhancement for Object Detection

Distributed and Efficient Object Detection Via Interactions among Devices, Edge, and Cloud

An adaptive offloading framework for license plate detection in collaborative edge and cloud computing

FrameHopper: Selective Processing of Video Frames in Detection-driven Real-Time Video Analytics

Edge Real-Time Object Detection and DPU-Based Hardware Implementation for Optical Remote Sensing Images

An Efficient Real-Time Object Detection Framework on Resource-Constricted Hardware Devices via Software and Hardware Co-design

RILOD: Near Real-Time Incremental Learning for Object Detection at the Edge