Review of Visual Reconstruction Methods of Retina-Like Vision Sensors

Lin ZHU,Yonghong TIAN

DOI: https://doi.org/10.1360/ssi-2021-0397

2023-01-01

Scientia Sinica Informationis

Abstract:Retina-like vision sensors have been a new development in recent years. They output an asynchronous spike or event signal by continuously sampling the scene light intensity information. This research summarizes the sampling principle of existing retina-like vision sensors（including spatial contrast, temporal contrast, and temporal integration vision sensors） and retina-like cameras with additional circuits. Compared to traditional cameras, retina-like vision sensors have high dynamic ranges and high temporal domain resolution; however,as the spike signal is stored as 1 and 0（or-1）, compatibility with traditional vision is difficult to obtain. If neuromorphic vision sampling can be considered as the encoding process of the scene light intensity, then the visual information reconstruction is the decoding process. Thus, visual information reconstruction can bridge the gap between neuromorphic vision and traditional frame-based vision. Recently, many image reconstruction algorithms have used asynchronous spikes or event signals. By processing such spikes or event signals, an image of high temporal domain resolution can be reconstructed. In this research, according to the sampling characteristics of different retina-like vision sensors, we summarize the existing image reconstruction methods based on event and spike cameras. Moreover, the opportunities and challenges of retina-like visual sampling and visual information reconstruction, as well as their possible development directions, are discussed and prospected.