Abstract:Traditional computer-vision technology that relies on image sensors coupled with cloud processing or on-chip Artificial Intelligence (AI) processors have encountered significant challenges in terms of power consumption, delays arising from data transmission, and/or memory access. In-sensor and near-sensor computing have been reported to solve this issue by applying pixel or array level feature extraction [2–6]. In [2, 4], capacitors are utilized for analog-domain Haar filtering to reduce the processing power consumption, but sacrificing the Fill Factor (FF) due to the use of the capacitor array [2], or complicated pixel-level logic [4]. An image sensor in introduced in [3] with Hog feature-based object detection, achieving both low power and high accuracy for the detection of up to three object classes. However, it does not work on more complex tasks. Convolutional Neural Networks (CNNs) [5, 6] offer enhanced accuracy in practical applications, but increase the power consumption and circuit complexity. Optical-domain processing [7] is capable of parallel information processing through an optical architecture based on free space. However, in [7] this is based on diffraction and requires coherent light as input, which constrains the practicality for edge computing. This paper presents a Pulse Width Modulation (PWM) pixel-based image sensor array that integrates an optical-electronic hybrid 3-layer convolutional processing unit. The first layer of the convolution is performed in the optical domain without introducing extra power consumption. The integration of optical convolution with electrical convolution offers the potential to leverage the energy-efficient advantages of optical convolution, while simultaneously providing the essential programmability needed to address multiple use-cases through reprogramming within a single device. A dual-mode Processing Element (PE) is proposed enables both acquisition mode and calculation mode. The integration of a sum/differential dual-counter design and the incorporation of two different pixel reset methods are employed in the co-design of PWM pixels and PEs to effectively mitigate computational power consumption.

A Multimode Neuromorphic Vision Sensor with Improved Brightness Measurement Performance by Pulse Coding Method

A Multi-mode Neuromorphic Vision Sensor With Improved Brightness Measurement Performance by Pulse Coding Method

100 Ghz Integrated Cmos Passive Imager with > 100 Mv/W Responsivity, 23fw/root Hz Nep

A Single-Slope PWM Imaging Method for Multi-Mode Dynamic Vision Sensor

A 128$\times$ 128 120 dB 15 $\mu$s Latency Asynchronous Temporal Contrast Vision Sensor

A Single Chip SPAD Based Vision Sensing System With Integrated Memristive Spiking Neuromorphic Processing

Computational event-driven vision sensors for in-sensor spiking neural networks

A Dual-Mode Image Sensor Using an All-Inorganic Perovskite Nanowire Array for Standard and Neuromorphic Imaging

Multiframe-integrated, In-Sensor Computing Using Persistent Photoconductivity

A Single-Pixel Event Photoactive Device for Real-Time, In-Sensor Spatiotemporal Optical Information Processing

A 128$\,\times$ 128 1.5% Contrast Sensitivity 0.9% FPN 3 µs Latency 4 mW Asynchronous Frame-Free Dynamic Vision Sensor Using Transimpedance Preamplifiers

6.9 A 0.35V 0.367TOPS/W Image Sensor with 3-Layer Optical-Electronic Hybrid Convolutional Neural Network

A flexible ultrasensitive optoelectronic sensor array for neuromorphic vision systems

A 2.17μw@120fps Ultra-Low-Power Dual-Mode CMOS Image Sensor with Senputing Architecture

Analysis and Design of a CMOS Ultra-High-Speed Burst Mode Imager with In-Situ Storage Topology Featuring In-Pixel CDS Amplification

Monolithic 2D Perovskites Enabled Artificial Photonic Synapses for Neuromorphic Vision Sensors

Nearly Panoramic Neuromorphic Vision with Transparent Photosynapses

An improved pixel circuit with low noise event rate and enhanced bright‐light sensitivity for dynamic vision sensor

Ultrasensitive dim-light neuromorphic vision sensing via momentum-conserved reconfigurable van der Waals heterostructure

Pixel-wise programmability enables dynamic high-SNR cameras for high-speed microscopy

A Joint Intensity-Neuromorphic Event Imaging System With Bandwidth-Limited Communication Channel