Abstract:Video surveillance systems are integral to bolstering safety and security across multiple settings. With the advent of deep learning (DL), a specialization within machine learning (ML), these systems have been significantly augmented to facilitate DL-based video surveillance services with notable precision. Nevertheless, DL-based video surveillance services, which necessitate the tracking of object movement and motion tracking (e.g., to identify unusual object behaviors), can demand a significant portion of computational and memory resources. This includes utilizing GPU computing power for model inference and allocating GPU memory for model loading. To tackle the computational demands inherent in DL-based video surveillance, this study introduces a novel video surveillance management system designed to optimize operational efficiency. At its core, the system is built on a two-tiered edge computing architecture (i.e., client and server through socket transmission). In this architecture, the primary edge (i.e., client side) handles the initial processing tasks, such as object detection, and is connected via a Universal Serial Bus (USB) cable to the Closed-Circuit Television (CCTV) camera, directly at the source of the video feed. This immediate processing reduces the latency of data transfer by detecting objects in real time. Meanwhile, the secondary edge (i.e., server side) plays a vital role by hosting a dynamically controlling threshold module targeted at releasing DL-based models, reducing needless GPU usage. This module is a novel addition that dynamically adjusts the threshold time value required to release DL models. By dynamically optimizing this threshold, the system can effectively manage GPU usage, ensuring resources are allocated efficiently. Moreover, we utilize federated learning (FL) to streamline the training of a Long Short-Term Memory (LSTM) network for predicting imminent object appearances by amalgamating data from diverse camera sources while ensuring data privacy and optimized resource allocation. Furthermore, in contrast to the static threshold values or moving average techniques used in previous approaches for the controlling threshold module, we employ a Deep Q-Network (DQN) methodology to manage threshold values dynamically. This approach efficiently balances the trade-off between GPU memory conservation and the reloading latency of the DL model, which is enabled by incorporating LSTM-derived predictions as inputs to determine the optimal timing for releasing the DL model. The results highlight the potential of our approach to significantly improve the efficiency and effective usage of computational resources in video surveillance systems, opening the door to enhanced security in various domains.

Data Processing Unit for Energy Saving in Computer Vision: Weapon Detection Use Case

Characterization of Real-Time Object Detection Workloads on Vehicular Edge

AyE-Edge: Automated Deployment Space Search Empowering Accuracy Yet Efficient Real-Time Object Detection on the Edge

Survey on Energy-Efficient Deep Neural Networks for Computer Vision

ADDP: Anomaly Detection for DTU Based on Power Consumption Side-Channel

Energy-Efficient Real-Time UAV Object Detection on Embedded Platforms

Reduced CPU Workload for Human Pose Detection with the Aid of a Low-Resolution Infrared Array Sensor on Embedded Systems

A power-aware vision-based virtual sensor for real-time edge computing

Low-cost system for real-time verification of personal protective equipment in industrial facilities using edge computing devices

DPDPU: Data Processing with DPUs

Cloud-assisted analysis for energy efficiency in intelligent video systems

Deep Reinforcement Learning-Empowered Cost-Effective Federated Video Surveillance Management Framework

CNN Inference acceleration using low-power devices for human monitoring and security scenarios

Application of Machine Vision Techniques in Low-Cost Devices to Improve Efficiency in Precision Farming

Benchmarking Edge Computing Devices for Grape Bunches and Trunks Detection using Accelerated Object Detection Single Shot MultiBox Deep Learning Models

Beyond Inference: Performance Analysis of DNN Server Overheads for Computer Vision

Using Physical Dynamics: Accurate and Real-Time Object Detection for High-Resolution Video Streaming on Internet of Things Devices

Energy-Efficient & Real-Time Computer Vision with Intelligent Skipping via Reconfigurable CMOS Image Sensors

Exploring the Vision Processing Unit as Co-processor for Inference

Computer Vision on the Edge to Reduce Network Bandwidth Consumption and Computing Resources in Multi-view 3D Industrial Inspection without Hidden Surfaces

A methodological framework for optimizing the energy consumption of deep neural networks: a case study of a cyber threat detector