Abstract:Modern-day life is driven by electronic devices connected to the internet. The emerging research field of the Internet-of-Things (IoT) has become popular, just as there has been a steady increase in the number of connected devices. Since many of these devices are utilised to perform CV tasks, it is essential to understand their power consumption against performance. We report the power consumption profile and analysis of the NVIDIA Jetson Nano board while performing object classification. The authors present an extensive analysis regarding power consumption per frame and the output in frames per second using YOLOv5 models. The results show that the YOLOv5n outperforms other YOLOV5 variants in terms of throughput (i.e. 12.34 fps) and low power consumption (i.e. 0.154 mWh/frame).

What problem does this paper attempt to address?

The paper aims to address the energy consumption issues of edge devices when performing artificial intelligence inference tasks. Specifically, the research objectives include the following aspects: 1. **Energy Consumption Analysis**: Evaluate the energy consumption of the NVIDIA Jetson Nano board when performing object classification tasks. By using different variants of the YOLOv5 model (such as YOLOv5n, YOLOv5s, etc.), the researchers conducted a detailed analysis of the energy required to process each frame of the image. 2. **Performance and Energy Consumption Trade-off**: Explore the trade-off relationship between energy consumption and performance for different scales of the YOLOv5 model. The study found that the smaller YOLOv5n model performs best in terms of energy consumption (0.154 mWh/frame), while the larger YOLOv5x model, although more accurate, significantly increases energy consumption (2.5 mWh/frame). 3. **Advantages of Heterogeneous Computing**: By comparing the scenarios of using only the CPU and combining CPU and GPU for inference, it was found that combining CPU and GPU resources can significantly improve performance and reduce energy consumption. For example, YOLOv5n can achieve a throughput of 12 fps in CPU+GPU mode, while only 0.4 fps when using only the CPU. 4. **Comparison of Edge Computing and Cloud Computing**: The study also compared the differences in energy consumption and performance between edge computing (using Jetson Nano) and cloud computing (using Azure services). The results show that in some cases, edge computing not only has lower energy consumption but also provides higher data throughput and lower latency. In summary, this paper provides important insights into the trade-off between energy consumption and performance when performing AI inference tasks on edge devices by testing the energy consumption and performance of different variants of the YOLOv5 model on the NVIDIA Jetson Nano board. This information is crucial for making informed decisions when deploying AI systems in resource-constrained environments.

Estimating the Power Consumption of Heterogeneous Devices when performing AI Inference

MOC: Multi-Objective Mobile CPU-GPU Co-Optimization for Power-Efficient DNN Inference

DACO: Pursuing Ultra-low Power Consumption Via DNN-Adaptive CPU-GPU CO-optimization on Mobile Devices

Real-time Object Detection Performance Analysis Using YOLOv7 on Edge Devices

Energy Consumption of Neural Networks on NVIDIA Edge Boards: an Empirical Model

Data-driven Software-based Power Estimation for Embedded Devices

Benchmarking Deep Learning Models for Object Detection on Edge Computing Devices

Image Classification on IoT Edge Devices: Profiling and Modeling

PowerNet: Learning-Based Real-Time Power-Budget Rendering

Analysing Edge Computing Devices for the Deployment of Embedded AI

Power Efficient Machine Learning Models Deployment on Edge IoT Devices

CNN Inference acceleration using low-power devices for human monitoring and security scenarios

Performance/power assessment of CNN packages on embedded automotive platforms

Benchmarking Deep Learning Models on NVIDIA Jetson Nano for Real-Time Systems: An Empirical Investigation

Characterizing Resource Heterogeneity in Edge Devices for Deep Learning Inferences

Optimizing convolutional neural networks for IoT devices: performance and energy efficiency of quantization techniques

A Deep Learning Framework Performance Evaluation to Use YOLO in Nvidia Jetson Platform

Benchmarking Edge Computing Devices for Grape Bunches and Trunks Detection using Accelerated Object Detection Single Shot MultiBox Deep Learning Models

PreVIous: A Methodology for Prediction of Visual Inference Performance on IoT Devices

Reaching for the Sky: Maximizing Deep Learning Inference Throughput on Edge Devices with AI Multi-Tenancy

Exploring the Boundaries of On-Device Inference: When Tiny Falls Short, Go Hierarchical