Abstract:The astounding development of optical sensing imaging technology, coupled with the impressive improvements in machine learning algorithms, has increased our ability to understand and extract information from scenic events. In most cases, Convolution neural networks (CNNs) are largely adopted to infer knowledge due to their surprising success in automation, surveillance, and many other application domains. However, the convolution operations' overwhelming computation demand has somewhat limited their use in remote sensing edge devices. In these platforms, real-time processing remains a challenging task due to the tight constraints on resources and power. Here, the transfer and processing of non-relevant image pixels act as a bottleneck on the entire system. It is possible to overcome this bottleneck by exploiting the high bandwidth available at the sensor interface by designing a CNN inference architecture near the sensor. This paper presents an attention-based pixel processing architecture to facilitate the CNN inference near the image sensor. We propose an efficient computation method to reduce the dynamic power by decreasing the overall computation of the convolution operations. The proposed method reduces redundancies by using a hierarchical optimization approach. The approach minimizes power consumption for convolution operations by exploiting the Spatio-temporal redundancies found in the incoming feature maps and performs computations only on selected regions based on their relevance score. The proposed design addresses problems related to the mapping of computations onto an array of processing elements (PEs) and introduces a suitable network structure for communication. The PEs are highly optimized to provide low latency and power for CNN applications. While designing the model, we exploit the concepts of biological vision systems to reduce computation and energy. We prototype the model in a Virtex UltraScale+ FPGA and implement it in Application Specific Integrated Circuit (ASIC) using the TSMC 90nm technology library. The results suggest that the proposed architecture significantly reduces dynamic power consumption and achieves high-speed up surpassing existing embedded processors' computational capabilities.

SAPPHIRE: an Always-on Context-Aware Computer Vision System for Portable Devices

AyE-Edge: Automated Deployment Space Search Empowering Accuracy Yet Efficient Real-Time Object Detection on the Edge

Senputing: An Ultra-Low-Power Always-On Vision Perception Chip Featuring the Deep Fusion of Sensing and Computing

HARP: Hierarchical Attention Oriented Region-Based Processing for High-Performance Computation in Vision Sensor

Energy-Efficient & Real-Time Computer Vision with Intelligent Skipping via Reconfigurable CMOS Image Sensors

Ultra-Efficient On-Device Object Detection on AI-Integrated Smart Glasses with TinyissimoYOLO

Multiframe-integrated, In-Sensor Computing Using Persistent Photoconductivity

CloudEye: A New Paradigm of Video Analysis System for Mobile Visual Scenarios

A Computer Vision Sensor for Efficient Object Detection Under Varying Lighting Conditions

PicSys: Energy-Efficient Fast Image Search on Distributed Mobile Networks

Speck: A Smart event-based Vision Sensor with a low latency 327K Neuron Convolutional Neuronal Network Processing Pipeline

A 4.57 Μw@120fps Vision System of Sensing with Computing for BNN-Based Perception Applications

Low-Latency Proactive Continuous Vision.

Redesigning the Wearable Camera Pipeline to Detect Key Body Parts at Low Power

A power-aware vision-based virtual sensor for real-time edge computing

Real-time Object Detection Towards High Power Efficiency.

An Analog-Memoryless Near Sensor Computing Architecture for Always-On Intelligent Perception Applications

MOSAIC: Spatially-Multiplexed Edge AI Optimization over Multiple Concurrent Video Sensing Streams

A low-power end-to-end hybrid neuromorphic framework for surveillance applications

Towards an Efficient CNN Inference Architecture Enabling In-Sensor Processing

NS-MD: Near-Sensor Motion Detection with Energy Harvesting Image Sensor for Always-On Visual Perception.