Abstract:Traditional computer-vision technology that relies on image sensors coupled with cloud processing or on-chip Artificial Intelligence (AI) processors have encountered significant challenges in terms of power consumption, delays arising from data transmission, and/or memory access. In-sensor and near-sensor computing have been reported to solve this issue by applying pixel or array level feature extraction [2–6]. In [2, 4], capacitors are utilized for analog-domain Haar filtering to reduce the processing power consumption, but sacrificing the Fill Factor (FF) due to the use of the capacitor array [2], or complicated pixel-level logic [4]. An image sensor in introduced in [3] with Hog feature-based object detection, achieving both low power and high accuracy for the detection of up to three object classes. However, it does not work on more complex tasks. Convolutional Neural Networks (CNNs) [5, 6] offer enhanced accuracy in practical applications, but increase the power consumption and circuit complexity. Optical-domain processing [7] is capable of parallel information processing through an optical architecture based on free space. However, in [7] this is based on diffraction and requires coherent light as input, which constrains the practicality for edge computing. This paper presents a Pulse Width Modulation (PWM) pixel-based image sensor array that integrates an optical-electronic hybrid 3-layer convolutional processing unit. The first layer of the convolution is performed in the optical domain without introducing extra power consumption. The integration of optical convolution with electrical convolution offers the potential to leverage the energy-efficient advantages of optical convolution, while simultaneously providing the essential programmability needed to address multiple use-cases through reprogramming within a single device. A dual-mode Processing Element (PE) is proposed enables both acquisition mode and calculation mode. The integration of a sum/differential dual-counter design and the incorporation of two different pixel reset methods are employed in the co-design of PWM pixels and PEs to effectively mitigate computational power consumption.

Design of Switched-Current Based Low-Power PIM Vision System for IoT Applications

An 8-Bit in Resistive Memory Computing Core with Regulated Passive Neuron and Bitline Weight Mapping

A Low-Power In-Memory Multiplication and Accumulation Array with Modified Radix-4 Input and Canonical Signed Digit Weights

A Reconfigurable Convolution-in-Pixel CMOS Image Sensor Architecture

NS-CIM: A Current-Mode Computation-in-Memory Architecture Enabling Near-Sensor Processing for Intelligent IoT Vision Nodes.

A 1.8mW Perception Chip with Near-Sensor Processing Scheme for Low-Power AIoT Applications

Processing Near Sensor Architecture in Mixed-Signal Domain with CMOS Image Sensor of Convolutional-Kernel-Readout Method

A 4.57 Μw@120fps Vision System of Sensing with Computing for BNN-Based Perception Applications

A 5.9μw Ultra-Low-Power Dual-Resolution CIS Chip of Sensing-with-Computing for Always-on Intelligent Visual Devices

A Heterogeneous Microprocessor Based on All-Digital Compute-in-Memory for End-to-End AIoT Inference

An Energy-Efficient Computing-in-Memory NN Processor with Set-Associate Blockwise Sparsity and Ping-Pong Weight Update

AIMMI: Audio and Image Multi-Modal Intelligence Via a Low-Power SoC with 2-Mbyte On-Chip MRAM for IoT Devices

PIMulator-NN: an Event-Driven, Cross-level Simulation Framework for Processing-In-Memory Based Neural Network Accelerators

A 41.7TOPS/W@INT8 Computing-in-Memory Processor with Zig-Zag Backbone-Systolic CIM and Block/Self-Gating CAM for NN/Recommendation Applications

An Energy-Efficient Computing-in-Memory Neuromorphic System with On-Chip Training.

Ultra-low Power Always-on Intelligent and Connected SNN-based System for Multimedia IoT-enabled Applications

A Low-Power Hybrid-Precision Neuromorphic Processor with INT8 Inference and INT16 Online Learning in 40-Nm CMOS

Reliability-Aware Training and Performance Modeling for Processing-In-Memory Systems

A 0.8 V Intelligent Vision Sensor With Tiny Convolutional Neural Network and Programmable Weights Using Mixed-Mode Processing-in-Sensor Technique for Image Classification

6.9 A 0.35V 0.367TOPS/W Image Sensor with 3-Layer Optical-Electronic Hybrid Convolutional Neural Network

pPIM: A Programmable Processor-in-Memory Architecture With Precision-Scaling for Deep Learning