Abstract:The need for real-time processing fast moving objects in machine vision requires the cooperation of high frame rate camera and a large amount of computing resources. The cost, high detection bandwidth requirements, data and computation burden limit the wide applications of high frame rate machine vision. Compressive Video Sensing (CVS) allows capturing events at much higher frame rate with the slow camera, by reconstructing a frame sequence from a coded single image. At the same time, complex frame sequence reconstruction algorithms in CVS pose challenges for computing resources. Even though the reconstruction process is low computational complexity, image-dependent machine vision algorithms also suffers from a large amount of computing energy consumption. Here we present a novel CVS camera termed Temporal Ghost Fourier Compressive Inference Camera (TGIC), which provides a framework to minimize the data and computational burden simultaneously. The core of TGIC is co-design CVS encoding and machine vision algorithms both in optical domain. TGIC acquires pixel-wise temporal Fourier spectrum in one frame, and applies simple inverse fast Fourier transform algorithm to get the desired video. By implementing pre-designed optical Fourier sampling schemes, specific machine vision tasks can be accomplished in optical domain. In fact, the data captured by TGIC is the results of traditional machine vision algorithms derived from the video, therefore the computation resources will be greatly saved. In the experiments, we can recover 160 frames in 0.1s single exposure with 16x frame rate gain (periodic motion up to 2002 frames, 1000x frame rate gain), and typical machine vision applications such as moving object detection, tracking and extraction are also demonstrated.

Broad Dual-Band Temporal Compressive Imaging With Optical Calibration

Convolutional Neural Network Based Computational Imaging Spectroscopy

High-resolution Fast Mid-Wave Infrared Compressive Imaging.

On-the-fly Extrinsic Calibration of Multimodal Sensing System for Fast 3D Thermographic Scanning

Temporal Compressive Imaging For Video

Single-camera Full-Range High-Resolution Spectral Domain Optical Coherence Tomography

Snapshot Compressive ToF+Spectral Imaging via Optimized Color-Coded Apertures

Compressive Imaging and Spectroscopy – Beyond the Single Pixel Camera

High Throughput Dual-Wavelength Temperature Distribution Imaging Via Compressive Imaging

Dual-compressed Photoacoustic Single-Pixel Imaging

Compressive Whiskbroom Sensing and Data Reconstruction for Hyperspectral Imaging

Temporal Compressed Measurements for Block-wise Compressive Imaging

Temporal Compressive Complex Amplitude Imaging Based on Double Random Phase Encoding

Wide-field mid-infrared hyperspectral imaging beyond video rate

Complementary Compressive Imaging for the Telescopic System

Temporal Ghost Fourier Compressive Inference Camera

Compressive Sensing Based High-Speed Time-Stretch Optical Microscopy for Two-Dimensional Image Acquisition

Applications of Compressive Sensing in Optical Imaging

Image reconstruction of dynamic infrared single-pixel imaging system

Adaptive Compressive Sampling for Mid-infrared Spectroscopic Imaging

High-resolution Infrared or Millimeter Wave Imaging Based on Compressed Sensing