A Three-Wafer-Stacked Hybrid 15-MPixel CIS + 1-MPixel EVS With 4.6-GEvent/s Readout, In-Pixel TDC, and On-Chip ISP and ESP Function
Qing Qin,Jason Yang,Y. Kudo,Qiping Huang,Bo Mu,Bo Jiang,Wenlei Yang,Shoushun Chen,Shimpei Fukuoka,M. Uchiyama,Xueqing Wang,Peiwen Jiang,Peter Bartkovjak,TJ Dai,Hiroaki Ebihara,M. Guo,Dahai Zhou,Zhe Gao,Xiaoqin Hu,Huan Chen,Chengcheng Xu,Andreas Suess
DOI: https://doi.org/10.1109/JSSC.2023.3303154
IF: 5.4
2023-11-01
IEEE Journal of Solid-State Circuits
Abstract:We introduce a hybrid 4096 $\times $ 3680 CMOS image sensor (CIS) with an embedded 1032 $\times $ 928 event-based vision sensor (EVS). Within a four-by-four cluster of CIS pixels, one color channel is substituted to provide a photocurrent for an EVS pixel. The EVS readout circuit is allocated on a second wafer and wrapped behind the four-by-four pixel cluster. A third wafer is used to integrate all peripheral readout circuitry for the EVS and CIS functions while achieving a compact sensor footprint. The EVS pixel achieves sub-1 Hz noise rates and 3% contrast non-uniformity (CTNU) at a linear nominal contrast threshold (NCT) of 15%. The row and column readout circuitry bypasses elements without events in a fraction of a clock cycle. Furthermore, a skip function is realized that reads only bordering pixels of a connected set of events. This achieves an event rate of up to 4.6 GEvents/s (GEps). The sensor captures in-pixel time stamps through sampling of a gray code timer realizing a time-to-digital conversion (TDC). Image and event signal processors (ISP & ESP) are integrated on-chip. The ESP provides global ambient and global activity monitoring functions. The sensor supports various flicker mitigation approaches such as adjustment of contrast threshold, in-pixel bandpass filter function and region-of-interest (ROI), and deterministic or random subsampling. The hybrid sensor mitigates challenges in synchronization and parallax error compared with a two-sensor solution using a dedicated CIS and dedicated EVS sensor. Furthermore, only one package and lens are required reducing footprint and cost.
Physics,Computer Science,Engineering