Abstract:In the era of Artificial Intelligence (AI), bio-inspired perceptual computing system design brings favorable opportunities, while still facing considerable challenges in the meantime. Especially for tasks of image recognition in power-limited vision-based Internet of Things (IoT) devices, energy constraints due to the end of Dennard scaling limit the performance of Neural Network (NN) algorithms on popular digital platforms, which would not reach the energy efficiency requirement for embedded AI applications. In this paper, a processing near sensor architecture in mixed-signal domain with CMOS Image Sensor (CIS) of convolutional-kernel-readout method is proposed. Visual data is collected from a smart CIS, which can realize maximum $5 \times 5$ kernel-readout with minimum one slide step for convolutional operations. The outputs of CIS are directly processed by analog processing units locating near CIS without the constraint of digital clock and bottleneck of Analog-to-Digital Converter (ADC). By analyzing the effects of analog noise on classification accuracy, we further evaluate the fault-tolerance of the system to circuit noise and the device imperfection, such as mismatch and process variation. A mixed-signal visual perception chip is fabricated with a $32\times32$ image sensor and a Binarized Neural Network (BNN) processing array integrated with SMIC 180nm standard CMOS mixed-signal process. Measurement results show up to 545.4 GOPS/W energy efficiency with 1.8mW power consumption taking the advantages of ADC-free processing architecture. This work provides a promising alternative for low-power vision-based IoT intelligent applications.

A Precision-Improved Processing Architecture of Physical Computing for Energy-Efficient SIFT Feature Extraction

A Reconfigurable Convolution-in-Pixel CMOS Image Sensor Architecture

Physical Computing With No Clock to Implement the Gaussian Pyramid of SIFT Algorithm

Physical Computing Circuit with No Clock to Establish Gaussian Pyramid of SIFT Algorithm

A Near-array Convolution Computing Scheme Based on WSe<sub>2</sub> Photodiode

An Ultra-Fast and Low-Power Design of Analog Circuit Network for DoG Pyramid Construction of SIFT Algorithm

Optical Convolution Based Computational Method for Low-Power Image Processing

ASP-SIFT: Using Analog Signal Processing Architecture to Accelerate Keypoint Detection of SIFT Algorithm

Implementation of Parallel Acceleration for Real-time Extraction of Visual Features.

A Fast and Power-Efficient Hardware Architecture for Visual Feature Detection in Affine-SIFT.

Multiframe-integrated, In-Sensor Computing Using Persistent Photoconductivity

Utilizing Direct Photocurrent Computation and 2D Kernel Scheduling to Improve In-Sensor-Processing Efficiency

A 83fps 1080P Resolution 354 Mw Silicon Implementation for Computing the Improved Robust Feature in Affine Space

Parallelization of Computing-Intensive Tasks of SIFT Algorithm on a Reconfigurable Architecture System

Processing Near Sensor Architecture in Mixed-Signal Domain with CMOS Image Sensor of Convolutional-Kernel-Readout Method

A Power Efficient Architecture with Optimized Parallel Memory Accessing for Feature Generation

An Analog-Memoryless Near Sensor Computing Architecture for Always-On Intelligent Perception Applications

A 5.9μw Ultra-Low-Power Dual-Resolution CIS Chip of Sensing-with-Computing for Always-on Intelligent Visual Devices

A Hardware Accelerator with Variable Pixel Representation & Skip Mode Prediction for Feature Point Detection Part of SIFT Algorithm.

CMOS Image Sensor Data-Readout Method for Convolutional Operations with Processing Near Sensor Architecture.

Scale Invariant Feature Transform Algorithm Based on a Reconfigurable Architecture System