Abstract:The need for real-time processing fast moving objects in machine vision requires the cooperation of high frame rate camera and a large amount of computing resources. The cost, high detection bandwidth requirements, data and computation burden limit the wide applications of high frame rate machine vision. Compressive Video Sensing (CVS) allows capturing events at much higher frame rate with the slow camera, by reconstructing a frame sequence from a coded single image. At the same time, complex frame sequence reconstruction algorithms in CVS pose challenges for computing resources. Even though the reconstruction process is low computational complexity, image-dependent machine vision algorithms also suffers from a large amount of computing energy consumption. Here we present a novel CVS camera termed Temporal Ghost Fourier Compressive Inference Camera (TGIC), which provides a framework to minimize the data and computational burden simultaneously. The core of TGIC is co-design CVS encoding and machine vision algorithms both in optical domain. TGIC acquires pixel-wise temporal Fourier spectrum in one frame, and applies simple inverse fast Fourier transform algorithm to get the desired video. By implementing pre-designed optical Fourier sampling schemes, specific machine vision tasks can be accomplished in optical domain. In fact, the data captured by TGIC is the results of traditional machine vision algorithms derived from the video, therefore the computation resources will be greatly saved. In the experiments, we can recover 160 frames in 0.1s single exposure with 16x frame rate gain (periodic motion up to 2002 frames, 1000x frame rate gain), and typical machine vision applications such as moving object detection, tracking and extraction are also demonstrated.

High speed two-dimensional temporal compressive sampling microscopic camera

3D Framing Camera System Based on Sample Cumulation Method

High speed single-pixel imaging via time domain compressive sampling

Sinusoidal Sampling Enhanced Compressive Camera for High Speed Imaging

Continuously streaming compressed high-speed photography using time delay integration

Driving Techniques for High Frame Rate Ccd Camera

Compressive Sensing Based High-Speed Time-Stretch Optical Microscopy for Two-Dimensional Image Acquisition

High-speed Real-Time Single-Pixel Microscopy Based on Fourier Sampling

Temporal compressive holographic microscopy based on alternating deep denoisers and total variation prior

Temporal multiplexing of the scientific grade camera for hyper-frame-rate imaging.

High-Speed Real-Time Image Compression Based on All-Optical Discrete Cosine Transformation

Temporal compressive super-resolution microscopy at frame rate of 1200 frames per second and spatial resolution of 100 nm

High-speed compressive wide-field fluorescence microscopy with an alternant deep denoisers-based image reconstruction algorithm

High-speed microscopic imaging system with incoherent broadband optical source

High-speed compressed-sensing fluorescence lifetime imaging microscopy of live cells

GHz Optical Time-Stretch Microscopy by Compressive Sensing

A Compact and Highly Efficient Compressive Sensing Imaging System Using In-Fiber Grating

High-Speed Compressive Microscopy of Flowing Cells Using Sinusoidal Illumination Patterns

Temporal Ghost Fourier Compressive Inference Camera

Time-encoded single-pixel 3D imaging

An Encoded Photodetector Array for High-frame-rate Compressed Sensing and Computational Imaging