Abstract:Ultra high-speed object location has potential prospects in civilian and scientific applications. However,the conventional image sensor has a bottleneck that can not achieve continuous ultra high-speed imaging. The Single-Photon Avalanche Diode (SPAD) image sensor,as a new type of spike image sensor,can realize continuous ultra high-speed imaging to support high-speed moving object location. The background subtraction method is a simple and effective high- speed location method,but the image noise caused by the SPAD image sensor will seriously interfere with the processing effect of the background subtraction method. Therefore,denoise processing is required before locating. This paper proposes an ultra high-speed object location processor for SPAD image sensors. It consists of a Processing Element(PE) array,a Y-feature generator,and a position calculator,and is capable of denoising and object location processing. The SPAD image sensor adopts row rolling exposure to produce N pixel output of one row at once exposure. The output format of the SPAD image sensor is a single-bit spike image containing only " 1"and" 0". The processing array contains N processing units,each PE unit contains 9 ALUs,which equals the size of filtering window 3x3,for processing Gaussian filtering and background subtraction methods with the current and earliest frames of spiking image data accessed from its two adjacent PEs. The current frame data,the earliest frame data,and the convolution kernel are calculated by complement logic and XOR gate in each ALU of the PE, and then the ALU results and the data in the memory are accumulated to obtain the filtering result. The object location module outputs a row of X feature vector and a column of Y feature vector at the end of each frame. The position calculator compares the feature vector with their thresholds and outputs the coordinate position of the object. We propose the Gaussian filtering and background subtraction method to remove the multiplication and subtraction operations,which reduces the computational complexity and hardware resource consumption while improving the processing speed significantly. Based on the overlapping sliding window method on impulse images, this paper replaces multiplications with additions merely by refactoring the filtering equation without any hardware overhead. During background subtraction,a static scene without objects is used as the background. The image is accumulated and the Gaussian filtering method is applied to obtain a background image. Then the negative value of the background image is regarded as the initial partial sum accumulation,and Gaussian filtering is applied to the following images containing moving objects based on this initial partial sum. In this way,we implement the background subtraction algorithm without a subtractor in the data path. Fixed pattern noise removal can be performed by simply controlling the relationship between the number of accumulated images of the foreground and background,and reusing the circuit of the background subtraction method to perform fixed pattern noise removal. The object location method determines the coordinates of the four corners by comparing the result of background subtraction with the threshold. The whole process is implemented on the FPGA development board. The experimental results show that the quality of the recovered grayscale image is significantly improved after Gaussian filtering. The processor can process spike images with a resolution of 128x128 at a speed of 100 Kfps, and locate moving objects in it. Compared with other studies focusing on ultra high-speed object location,the processor we proposed reaches a favorable trade- off between the processing algorithm complexity and hardware resource overhead. This paper also compares the performance of the processor with the original naive Gaussian filter circuit using the multiplication operation with our optimized processor. The result shows that our optimized processor can be synthesized to a frequency of 31.8% higher than the original one and achieves about 72.4% superiority over the original one on hardware resources. Moreover,our solution also can be applied to other spike image sensors to build a sensing-computing system.

SOPC-based Real-Time Spots Detection and Ordering for an Artificial Compound Eye of 3D Object Detection

Sad-Based Stereo Vision Machine on A System-On-Programmable-Chip (Sopc)

Infrared Small Target Tracking Based on Sopc

Development of an Artificial Compound Eye System for Three-Dimensional Object Detection

Arthropod Eye-Inspired Digital Camera With Unique Imaging Characteristics

Position detection with high precision using novel compound-eye sensor

SoPC implementation of autofocus for infrared imaging apparatus based on retinal vision mechanism

Real-time and Accurate Monocular 3D Sensor Using the Reference Plane Calibration and an Optimized SGM Based on Opencl Acceleration

Matching Between the Light Spots and Lenslets of an Artificial Compound Eye System

A Wide-Field and High-Resolution Lensless Compound Eye Microsystem for Real-Time Target Motion Perception

High-Speed Optical 3D Measurement Sensor for Industrial Application

A Compact Visible Bionic Compound Eyes System Based on Micro-Surface Fiber Faceplate

A planar compound eye based microsystem for high precision 3D perception

Real-time System for Small Target Detection and Tracking in Infrared Image

Vision Processing System for Chip Mounter Based on SOPC

An Ultra High-speed Object Location Processor for SPAD Spike Image Sensors

An artificial compound eye system for large field imaging

Stereoscopic artificial compound eyes for spatiotemporal perception in three-dimensional space

Structural Design and Image Processing of a Spherical Artificial Compound Eye

Speck: A Smart event-based Vision Sensor with a low latency 327K Neuron Convolutional Neuronal Network Processing Pipeline

An Opencl-Based Speckle Matching on the Monocular 3D Sensor Using Speckle Projection