Abstract:We introduce a hybrid 4096 $\times $ 3680 CMOS image sensor (CIS) with an embedded 1032 $\times $ 928 event-based vision sensor (EVS). Within a four-by-four cluster of CIS pixels, one color channel is substituted to provide a photocurrent for an EVS pixel. The EVS readout circuit is allocated on a second wafer and wrapped behind the four-by-four pixel cluster. A third wafer is used to integrate all peripheral readout circuitry for the EVS and CIS functions while achieving a compact sensor footprint. The EVS pixel achieves sub-1 Hz noise rates and 3% contrast non-uniformity (CTNU) at a linear nominal contrast threshold (NCT) of 15%. The row and column readout circuitry bypasses elements without events in a fraction of a clock cycle. Furthermore, a skip function is realized that reads only bordering pixels of a connected set of events. This achieves an event rate of up to 4.6 GEvents/s (GEps). The sensor captures in-pixel time stamps through sampling of a gray code timer realizing a time-to-digital conversion (TDC). Image and event signal processors (ISP & ESP) are integrated on-chip. The ESP provides global ambient and global activity monitoring functions. The sensor supports various flicker mitigation approaches such as adjustment of contrast threshold, in-pixel bandpass filter function and region-of-interest (ROI), and deterministic or random subsampling. The hybrid sensor mitigates challenges in synchronization and parallax error compared with a two-sensor solution using a dedicated CIS and dedicated EVS sensor. Furthermore, only one package and lens are required reducing footprint and cost.

A 39,000 Subexposures/s CMOS Image Sensor with Dual-tap Coded-exposure Data-memory Pixel for Adaptive Single-shot Computational Imaging

39 000-Subexposures/s Dual-ADC CMOS Image Sensor With Dual-Tap Coded-Exposure Pixels for Single-Shot HDR and 3-D Computational Imaging

A Reconfigurable Convolution-in-Pixel CMOS Image Sensor Architecture

A closed-loop, all-electronic pixel-wise adaptive imaging system for high dynamic range videography

A Dual-Exposure Wide Dynamic Range CMOS Image Sensor with 12bit Column-Parallel Incremental Sigma-Delta ADC

An Over 90 dB Intra-Scene Single-Exposure Dynamic Range CMOS Image Sensor Using a 3.0 μm Triple-Gain Pixel Fabricated in a Standard BSI Process

A 316MP, 120FPS, High Dynamic Range CMOS Image Sensor for Next Generation Immersive Displays

A 1920 $\times$ 1080 Array 2-D/3-D Image Sensor with 3- $\mu$ S Row-Time Single-Slope ADC and 100-Mhz Demodulated PPD Locked-In Pixel

An Encoded Photodetector Array for High-frame-rate Compressed Sensing and Computational Imaging

A 3.0 µm Pixels and 1.5 µm Pixels Combined Complementary Metal-Oxide Semiconductor Image Sensor for High Dynamic Range Vision beyond 106 dB

A closed-loop all-electronic pixel-wise adaptive imaging system for high dynamic range video

Multiresolution image sensor

A Dynamic Range Extension Technique for CMOS Image Sensors With In-Pixel Dual Exposure Synthesis

Analysis and Design of a CMOS Ultra-High-Speed Burst Mode Imager with In-Situ Storage Topology Featuring In-Pixel CDS Amplification

CMOS Image Sensor Data-Readout Method for Convolutional Operations with Processing Near Sensor Architecture.

Wide Dynamic Range Cmos Image Sensor with In-Pixel Double-Exposure and Synthesis

Simulations and Design of a Single-Photon CMOS Imaging Pixel Using Multiple Non-Destructive Signal Sampling

A Global Shutter High Speed TDI CMOS Image Sensor with Pipelined Charge Transfer Pixel

A Three-Wafer-Stacked Hybrid 15-MPixel CIS + 1-MPixel EVS With 4.6-GEvent/s Readout, In-Pixel TDC, and On-Chip ISP and ESP Function

A Low Power Global Shutter Pixel with Extended FD Voltage Swing Range for Large Format High Speed CMOS Image Sensor

A CMOS SPAD Sensor With a Multi-Event Folded Flash Time-to-Digital Converter for Ultra-Fast Optical Transient Capture