Abstract:Innovative enhancement in embedded system platforms, specifically hardware accelerations, significantly influence the application of deep learning in real-world scenarios. These innovations translate human labor efforts into automated intelligent systems employed in various areas such as autonomous driving, robotics, Internet-of-Things (IoT), and numerous other impactful applications. NVIDIA's Jetson platform is one of the pioneers in offering optimal performance regarding energy efficiency and throughput in the execution of deep learning algorithms. Previously, most benchmarking analysis was based on 2D images with a single deep learning model for each comparison result. In this paper, we implement an end-to-end video-based crime-scene anomaly detection system inputting from surveillance videos and the system is deployed and completely operates on multiple Jetson edge devices (Nano, AGX Xavier, Orin Nano). The comparison analysis includes the integration of Torch-TensorRT as a software developer kit from NVIDIA for the model performance optimisation. The system is built based on the PySlowfast open-source project from Facebook as the coding template. The end-to-end system process comprises the videos from camera, data preprocessing pipeline, feature extractor and the anomaly detection. We provide the experience of an AI-based system deployment on various Jetson Edge devices with Docker technology. Regarding anomaly detectors, a weakly supervised video-based deep learning model called Robust Temporal Feature Magnitude Learning (RTFM) is applied in the system. The approach system reaches 47.56 frames per second (FPS) inference speed on a Jetson edge device with only 3.11 GB RAM usage total. We also discover the promising Jetson device that the AI system achieves 15% better performance than the previous version of Jetson devices while consuming 50% less energy power.

Real-Time Motorbike Detection: AI on the Edge Perspective

Characterization of Real-Time Object Detection Workloads on Vehicular Edge

AyE-Edge: Automated Deployment Space Search Empowering Accuracy Yet Efficient Real-Time Object Detection on the Edge

Real-Time Embedded Implementation of Improved Object Detector for Resource-Constrained Devices

Embedded and Real-Time Vehicle Detection System for Challenging On-Road Scenes

Real-Time Pedestrian Detection on IoT Edge Devices: A Lightweight Deep Learning Approach

Real-time Object Detection Performance Analysis Using YOLOv7 on Edge Devices

Fast Object Detection with a Machine Learning Edge Device

Robust real-time pedestrian detection on embedded devices

Performance Evaluation of Real-Time Object Detection for Electric Scooters

Evaluation of Thermal Imaging on Embedded GPU Platforms for Application in Vehicular Assistance Systems

An Optimized DNN Model for Real-Time Inferencing on an Embedded Device

Real-Time Apple Detection System Using Embedded Systems With Hardware Accelerators: An Edge AI Application

Implementation of an Edge-Computing Vision System on Reduced-Board Computers Embedded in UAVs for Intelligent Traffic Management

Resource-Efficient Visual Multiobject Tracking on Embedded Device

Deep Learning-Based Real-Time Multiple-Object Detection and Tracking from Aerial Imagery via a Flying Robot with GPU-Based Embedded Devices

Towards Real-Time Object Detection on Embedded Systems.

Robust Real-time Pedestrian Detection in Aerial Imagery on Jetson TX2

Benchmarking Edge Computing Devices for Grape Bunches and Trunks Detection using Accelerated Object Detection Single Shot MultiBox Deep Learning Models

Benchmarking Jetson Edge Devices with an End-to-end Video-based Anomaly Detection System

Monocular Cyclist Detection with Convolutional Neural Networks