Abstract:Traditional computer-vision technology that relies on image sensors coupled with cloud processing or on-chip Artificial Intelligence (AI) processors have encountered significant challenges in terms of power consumption, delays arising from data transmission, and/or memory access. In-sensor and near-sensor computing have been reported to solve this issue by applying pixel or array level feature extraction [2–6]. In [2, 4], capacitors are utilized for analog-domain Haar filtering to reduce the processing power consumption, but sacrificing the Fill Factor (FF) due to the use of the capacitor array [2], or complicated pixel-level logic [4]. An image sensor in introduced in [3] with Hog feature-based object detection, achieving both low power and high accuracy for the detection of up to three object classes. However, it does not work on more complex tasks. Convolutional Neural Networks (CNNs) [5, 6] offer enhanced accuracy in practical applications, but increase the power consumption and circuit complexity. Optical-domain processing [7] is capable of parallel information processing through an optical architecture based on free space. However, in [7] this is based on diffraction and requires coherent light as input, which constrains the practicality for edge computing. This paper presents a Pulse Width Modulation (PWM) pixel-based image sensor array that integrates an optical-electronic hybrid 3-layer convolutional processing unit. The first layer of the convolution is performed in the optical domain without introducing extra power consumption. The integration of optical convolution with electrical convolution offers the potential to leverage the energy-efficient advantages of optical convolution, while simultaneously providing the essential programmability needed to address multiple use-cases through reprogramming within a single device. A dual-mode Processing Element (PE) is proposed enables both acquisition mode and calculation mode. The integration of a sum/differential dual-counter design and the incorporation of two different pixel reset methods are employed in the co-design of PWM pixels and PEs to effectively mitigate computational power consumption.

A variable threshold visual sensing and image reconstruction method based on pulse sequence

A Reconfigurable Convolution-in-Pixel CMOS Image Sensor Architecture

Transverse Dynamic Modulation Transfer Function of Subpixel Push-Scanning Imaging by Linear CCD

Ccd High Speed Videography System With New Concepts And Techniques

Image Lag Elimination Algorithm for Pulse-Sequence-Based High-Speed Image Sensor

A 1/2.5 Inch Vga 400 Fps Cmos Image Sensor with High Sensitivity for Machine Vision

An Intrascene Wide Dynamic Range CMOS Image Sensor Using Dual-Gain Ramp Generator for Machine Vision Applications

A Method of Biomimetic Visual Perception and Image Reconstruction Based on Pulse Sequence of Events

A Self-Synchronized High Speed Computational Ghost Imaging System: A Leap Towards Dynamic Capturing

Imaging Theory of a Retina-Like CMOS Sensor in High Speed Forward Motion

CMOS Image Sensor Data-Readout Method for Convolutional Operations with Processing Near Sensor Architecture.

Design and Simulation of an Application Specific CMOS Image Sensor

High-speed and high-precision image acquisition system based on FPGA and CIS

Fast high quality computational ghost imaging based on saliency variable sampling detection

A 5.9μw Ultra-Low-Power Dual-Resolution CIS Chip of Sensing-with-Computing for Always-on Intelligent Visual Devices

Study on the key technologies of a high-speed CMOS camera

Circuit Techniques to Improve Low-Light Characteristics and High-Accuracy Evaluation System for CMOS Image Sensor

An Encoded Photodetector Array for High-frame-rate Compressed Sensing and Computational Imaging

6.9 A 0.35V 0.367TOPS/W Image Sensor with 3-Layer Optical-Electronic Hybrid Convolutional Neural Network

Direct Noise-Resistant Edge Detection with Edge-Sensitive Single-Pixel Imaging Modulation

Image Acquisition System Using On Sensor Compressed Sampling Technique