A Bio-Inspired Spiking Vision Chip Based on SPAD Imaging and Direct Spike Computing for Versatile Edge Vision
Xu Yang,Chunhe Yao,Lei Kang,Qian Luo,Nan Qi,Runjiang Dou,Shuangming Yu,Peng Feng,Zhongming Wei,Jian Liu,Kaiyou Wang,Nanjian Wu,Liyuan Liu
DOI: https://doi.org/10.1109/jssc.2023.3340018
IF: 5.4
2024-01-01
IEEE Journal of Solid-State Circuits
Abstract:Edge vision devices face real 3-D scenes with lighting fluctuations. They require a vision chip capable of acquiring 2-D/3-D vision with a high dynamic range and performing real-time intelligent in situ processing, while maintaining low end-to-end latency. Aimed at this versatile edge vision, this article presents a bio-inspired full-spiking vision chip that consists of a single-photon-avalanche-diode (SPAD) image sensor and a spiking vision processor monolithically. To decrease latency while realizing versatile functions, we build a bio-inspired full spiking vision on chip by directly coupling spike-like SPAD imaging with a neuroinspired spike-computing paradigm. By modeling this spiking visual flow, we theoretically demonstrate direct coupling and derive a spike preprocessing method for dim-vision enhancement. The SPAD image sensor equipped with a configurable exposure gating scheme can realize three modes: 2-D, 3-D depth, and dim-vision imaging. The spiking vision processor contains a parallel processing element (PE) array and can be reconfigured as a preprocessor or a spiking neural network (SNN) processor. A prototype chip integrated with a $128\ttimes 128$ SPAD image sensor with a reconfigurable spiking vision processor (256 PEs) was implemented in a 0.18- $\mu$ m CMOS technology. The chip experimentally demonstrates three imaging modes, 99.30% classification accuracy on the MNIST dataset with 2-D vision, only 3.9% accuracy loss under 0.02 lux with dim-vision, 1.68 cm obstacle detection error with 3-D depth vision, and a 3.58 $\mu$ s latency for light adaptation. The chip realizes a 100 dB dynamic range, 100 000 spiking maps per second (SMps), and 81.92 GSOPS at an 80-MHz clock.