Mask-based single-pixel tracking and imaging for moving objects

Yu Zhang,Hongjie Wang,Yongkai Yin,Wenjie Jiang,Baoqing Sun

DOI: https://doi.org/10.1364/oe.501531

IF: 3.8

2023-09-15

Optics Express

Abstract:Yu Zhang, Hongjie Wang, Yongkai Yin, Wenjie Jiang, Baoqing Sun Tracking and imaging for high-speed moving objects have a wide range of application prospects in many fields, such as transportation and ... [Opt. Express 31, 32554-32564 (2023)]

optics

What problem does this paper attempt to address?

Fast and Sub-Pixel Precision Target Tracking Algorithm for Intelligent Dual-Resolution Camera.

Zhuang He,Qi Li,Huajun Feng,Zhihai Xu

DOI: https://doi.org/10.1007/s00371-019-01724-4

IF: 2.835

2019-01-01

The Visual Computer

Abstract:The intelligent dual-resolution camera can provide large imaging field and high-resolution for the parts in which we are interested simultaneously. It has important applications in visual tracking. Though many trackers show great robustness on recent benchmarks, few of them make high precision and run in real-time, which is harmful to practical applications. In this paper, we propose a fast and sub-pixel precision tracker. It uses the time-shift property of Fourier transform to convert the displacement of target in space domain into the period of the response in Fourier domain. Then an improved Hough transform is introduced to measure this period, so that the displacement with sub-pixel precision can be calculated. Furthermore, a method obtaining benchmarks utilizing the intelligent dual-resolution camera is proposed to verify the high-precision of our tracker. Many experiments have been done to compare the proposed tracker with some state-of-the-art sub-pixel precision trackers, and the results have shown that the proposed tracker outperform the best tracker by 15.4% in average median distance precision, while feasible for real-time tracking with the speed faster than 80 fps. What’s more, the proposed tracker have been evaluated on the dual-resolution camera, the results show that it can make the observation of targets in the high-resolution image more complete.
Multi-modal 3D Human Tracking for Robots in Complex Environment with Siamese Point-Video Transformer

Xin Shuo,Zhang Zhen,Wang Mengmeng,Hou Xiaojun,Guo Yaowei,Kang Xiao,Liu Liang,Liu Yong

DOI: https://doi.org/10.1109/icra57147.2024.10610979

2024-01-01

Abstract:Tracking a specific person in 3D scene is gaining momentum due to its numerous applications in robotics. Currently, most 3D trackers focus on driving scenarios with neglected jitter and uncomplicated surroundings, which results in their severe degeneration in complex environments, especially on jolting robot platforms (only 20-60% success rate). To improve the accuracy, a Point-Video-based Transformer Tracking model (PVTrack) is presented for robots. It is the first multi-modal 3D human tracking work that incorporates point clouds together with RGB videos to achieve information complementarity. Moreover, PVTrack proposes the Siamese Point-Video Transformer for feature aggregation to overcome dynamic environments, which captures more target-aware information through the hierarchical attention mechanism adaptively. Considering the violent shaking on robots and rugged terrains, a lateral Human-ware Proposal Network is designed together with an Anti-shake Proposal Compensation module. It alleviates the disturbance caused by complex scenes as well as the particularity of the robot platform. Experiments show that our method achieves state-of-the-art performance on both KITTI/Waymo datasets and a quadruped robot for various indoor and outdoor scenes.
Image-free real-time 3-D tracking of a fast-moving object using dual-pixel detection

Qiwen Deng,Zibang Zhang,Jingang Zhong

DOI: https://doi.org/10.1364/OL.399204

2020-09-01

Abstract:Real-time 3-D tracking of a fast-moving object has found important applications in industry, traffic control, sports, biomedicine, defense, etc. However, it is difficult to adopt typical image-based object tracking systems in a fast-moving object tracking in real time and for a long duration, because reliable and robust image processing and analysis algorithms are often computationally exhausted, and limited storage and bandwidth can hardly fulfill the great demand of high-speed photography. Here we report an image-free 3-D tracking approach. The approach uses only two single-pixel detectors and a high-speed spatial light modulator for data acquisition. By illuminating the target moving object with six single-period Fourier basis patterns, the approach is able to analytically calculate the position of the object with the corresponding single-pixel measurements. The approach is low-cost, and data- and computation-efficient. We experimentally demonstrate that the proposed approach can detect and track a fast-moving object at a frame rate of 1666 frames per second by using a 10,000 Hz digital micromirror device. Benefiting from the wide working spectrum of single-pixel detectors, the reported approach might be applicable for hidden fast-moving object tracking.
Single-pixel imaging of a translational object

Shijian Li,Cheng Yan,Yeliang Wang,Xu-ri Yao,Qi Zhao

DOI: https://doi.org/10.48550/arxiv.2209.01554

2022-01-01

Abstract:Image-free tracking methods based on single-pixel detectors (SPDs) can track a moving object at a very high frame rate, but they rarely can achieve simultaneous imaging of such an object. In this study, we propose a method for simultaneously obtaining the relative displacements and images of a translational object. Four binary Fourier patterns and two differential Hadamard patterns are used to modulate one frame of the object and then modulated light signals are obtained by SPD. The relative displacements and image of the moving object can be gradually obtained along with the detection. The proposed method does not require any prior knowledge of the object and its motion. The method has been verified by simulations and experiments, achieving a frame rate of 3332 Hz to acquire relative displacements of a translational object at a spatial resolution of $128 \times 128$ pixels using a 20000-Hz digital micro-mirror device. This proposed method can broaden the application of image-free tracking methods and obtain spatial information about moving objects.
Moving Object Tracking Research Based on Active Vision

Pen Feng,Wang Xuanyin,Wang Quanqi

DOI: https://doi.org/10.1109/wcica.2004.1342210

2004-01-01

Abstract:In an active vision condition both camera and object may move simultaneously and common motion detection methods could not deal with such situation. To overcome the shortcoming of common motion detection, a background-matching algorithm was proposed to solve such problem. When motion was detected, a Kalman predictor was used to estimate the object's position in the image. Combining increment PID controlling approach with dead zone, the camera can track the moving object steadily and reliably. The experiments proved the effectiveness of the algorithm.
Single-pixel Imaging of a Translational Object.

Shijian LI,Yan Cai,Yeliang Wang,Xu-Ri Yao,Qing Zhao

DOI: https://doi.org/10.1364/oe.481881

IF: 3.8

2023-01-01

Optics Express

Abstract:Image-free tracking methods based on single-pixel detectors (SPDs) can track a moving object at a very high frame rate, but they rarely can achieve simultaneous imaging of such an object. In this study, we propose a method for simultaneously obtaining the relative displacements and images of a translational object. Four binary Fourier patterns and two differential Hadamard patterns are used to modulate one frame of the object and then modulated light signals are obtained by SPD. The relative displacements and image of the moving object can be gradually obtained along with the detection. The proposed method does not require any prior knowledge of the object and its motion. The method has been verified by simulations and experiments, achieving a frame rate of 3332 Hz to acquire relative displacements of a translational object at a spatial resolution of 128 × 128 pixels using a 20000-Hz digital micro-mirror device. This proposed method can broaden the application of image-free tracking methods and obtain spatial information about moving objects.
Prior-free 3D tracking of a fast-moving object at 6667 frames per second with single-pixel detectors

Huan Zhang,Zonghao Liu,Mi Zhou,Zibang Zhang,Mu Ku Chen,Zihan Geng

DOI: https://doi.org/10.1364/ol.521176

IF: 3.6

2024-06-21

Optics Letters

Abstract:Huan Zhang, Zonghao Liu, Mi Zhou, Zibang Zhang, Muku Chen, Zihan Geng Real-time tracking and 3D trajectory computation of fast-moving objects is a promising technology, especially in the field of autonomous ... [Opt. Lett. 49, 3628-3631 (2024)]

optics
Tracking and fast imaging of a moving object via Fourier modulation

Shijian Li,Xu-Ri Yao,Wei Zhang,Yeliang Wang,Qing Zhao

DOI: https://doi.org/10.1103/physrevapplied.22.044007

IF: 4.6

2024-10-04

Physical Review Applied

Abstract:Recently, several single-pixel-imaging schemes have emerged for imaging fast-moving objects and have shown dramatic results. However, fast image reconstruction of a moving object with high quality is still challenging for single-pixel imaging, thereby limiting its practical application. In this pape... [Phys. Rev. Applied 22, 044007] Published Thu Oct 03, 2024

physics, applied
High-speed target tracking control system based on short-time rotational reflection imaging

Haoran Liu,Jiehu Kang,Luyuan Feng,Jianwei Zhang,Fei Fan,Zhen Zhang,Zefeng Sun,Bin Wu

DOI: https://doi.org/10.1364/ao.514800

IF: 1.9

2024-01-26

Applied Optics

Abstract:Haoran Liu, Jiehu Kang, Luyuan Feng, Jianwei Zhang, Fei Fan, Zhen Zhang, Zefeng Sun, Bin Wu High-speed tracking technology has wide applications in the military and aerospace industry. However, existing approaches, such as camera ... [Appl. Opt. 63, 1094-1104 (2024)]

optics
Lexicality and Modality Effects on Evoked Potentials in a Memory-Scanning Task

H PRATT

DOI: https://doi.org/10.1006/brln.1994.1019

IF: 2.781

1994-01-01

Brain and Language

Abstract:Single-pixel imaging enjoys advantages of low budget, broad spectrum, and high imaging speed. However, existing methods cannot clearly reconstruct the object that is fast rotating or randomly moving. In this work, we put forward an effective method to image a randomly moving object based on geometric moment analysis. To the best of our knowledge, this is the first work that reconstructs the shape and motion state of the target without prior knowledge of the speed or position. By using the cake-cutting order Hadamard illumination patterns and low-order geometric moment patterns, we obtain a high-quality video stream of the target which moves at high and varying translational and rotational speeds. The efficient method as verified by simulation and experimental results has great potential for practical applications such as Brownian motion microscopy and remote sensing.
Risley-prism-based tracking model for fast locating a target using imaging feedback

Anhu Li,Zusheng Zhao,Xingsheng Liu,Zhaojun Deng

DOI: https://doi.org/10.1364/OE.384694

IF: 3.8

2020-02-13

Optics Express

Abstract:Anhu Li, Zusheng Zhao, Xingsheng Liu, Zhaojun Deng Fast imaging tracking technology exhibits attractive application prospects in the emerging fields of target tracking and recognition. Smart ... [Opt. Express 28, 5378-5392 (2020)]

optics
Non-line-of-sight imaging and tracking of moving objects based on deep learning

JinHui He,ShuKong Wu,Ran Wei,YuNing Zhang

DOI: https://doi.org/10.1364/OE.455803

2022-05-09

Abstract:Detection of objects outside the line of sight remains a challenge in many practical applications. There have been various researches realizing 2D or 3D imaging of static hidden objects, whose aim are to improve the resolution of reconstructed images. While when it comes to the tracking of continuously moving objects, the speed of imaging and the accuracy of positioning becomes the priorities to optimize. Previous works have achieved centimeter-level or even higher precision of positioning through marking coordinates in intervals of 3 seconds to tens of milliseconds. Here a deep learning framework is proposed to realize the imaging and dynamic tracking of targets simultaneously using a standard RGB camera. Through simulation experiments, we firstly use the designed neural network to achieve positioning of a 3D mannequin with sub-centimeter accuracy (relative error under 1.8%), costing only 3 milliseconds per estimation in average. Furthermore, we apply the system to a physical scene to successfully recover the video signal of the moving target, intuitively revealing its trajectory. We demonstrate an efficient and inexpensive approach that can present the movement of objects around the corner in real time, profiting from the imaging of the NLOS scene, it is also possible to identify the hidden target. This technique can be ultilized to security surveillance, military reconnaissance, autonomous driving and other fields.
Gaze control system for tracking Quasi-1D high-speed moving object in complex background

Shuqiao Geng,Changan Zhu,Yi Jin,Lichao Wang,Hailong Tan

DOI: https://doi.org/10.1080/21642583.2022.2063204

2022-04-20

Abstract:A gaze control system for tracking Quasi-1D high-speed moving object is proposed, it can keep the object in the centre of the image within a certain range. Initially, the system structure is designed, and the tracking range of the system is expanded using a single saccade mirror. Then the model between the deflection angle of the saccade mirror and the pixel displacement is established. Finally, a frame-difference method based on image cropping is proposed to rapidly extract the moving object in the complex dynamic background. It feeds back the object position to the saccade mirror control system. The system adjusts the deflection angle of the saccade mirror in real time. Experimental results show that the system can satisfy the requirements of gaze control for tracking Quasi-1D high-speed moving object.
An Active Multi-Object Ultrafast Tracking System with CNN-Based Hybrid Object Detection

Qing Li,Shaopeng Hu,Kohei Shimasaki,Idaku Ishii

DOI: https://doi.org/10.3390/s23084150

2023-04-21

Abstract:This study proposes a visual tracking system that can detect and track multiple fast-moving appearance-varying targets simultaneously with 500 fps image processing. The system comprises a high-speed camera and a pan-tilt galvanometer system, which can rapidly generate large-scale high-definition images of the wide monitored area. We developed a CNN-based hybrid tracking algorithm that can robustly track multiple high-speed moving objects simultaneously. Experimental results demonstrate that our system can track up to three moving objects with velocities lower than 30 m per second simultaneously within an 8-m range. The effectiveness of our system was demonstrated through several experiments conducted on simultaneous zoom shooting of multiple moving objects (persons and bottles) in a natural outdoor scene. Moreover, our system demonstrates high robustness to target loss and crossing situations.
Dynamic Object Tracking for Self-Driving Cars Using Monocular Camera and LIDAR.

Lin Zhao,Meiling Wang,Sheng Su,Tong Liu,Yi Yang

DOI: https://doi.org/10.1109/iros45743.2020.9341179

2020-01-01

Abstract:The detection and tracking of dynamic traffic participants (e.g., pedestrians, cars, and bicyclists) plays an important role in reliable decision-making and intelligent navigation for autonomous vehicles. However, due to the rapid movement of the target, most current vision-based tracking methods, which perform tracking in the image domain or invoke 3D information in parts of their pipeline, have real-life limitations such as lack of the ability to recover tracking after the target is lost. In this work, we overcome such limitations and propose a complete system for dynamic object tracking in 3D space that combines: (1) a 3D position tracking algorithm based on monocular camera and LIDAR for the dynamic object; (2) a re-tracking mechanism (RTM) that restore tracking when the target reappears in camera's field of view. Compared with the existing methods, each sensor in our method is capable of performing its role to preserve reliability, and further extending its functions through a novel multimodality fusion module. We perform experiments in the real-world self-driving environment and achieve a desired 10Hz update rate for real-time performance. Our quantitative and qualitative analysis shows that this system is reliable for dynamic object tracking purposes of self-driving cars.
Single-shot 3D Tracking Based on Polarization Multiplexed Fourier-phase Camera

Jiajie Teng,Chengyang Hu,Honghao Huang,Minghua Chen,Sigang Yang,Hongwei Chen

DOI: https://doi.org/10.1364/prj.432292

IF: 7.6

2021-01-01

Photonics Research

Abstract:For moving objects, 3D mapping and tracking has found important applications in the 3D reconstruction for vision odometry or simultaneous localization and mapping. This paper presents a novel camera architecture to locate the fast-moving objects in four-dimensional (4D) space ( x , y , z , t ) through a single-shot image. Our 3D tracking system records two orthogonal fields-of-view (FoVs) with different polarization states on one polarization sensor. An optical spatial modulator is applied to build up temporal Fourier-phase coding channels, and the integration is performed in the corresponding CMOS pixels during the exposure time. With the 8 bit grayscale modulation, each coding channel can achieve 256 times temporal resolution improvement. A fast single-shot 3D tracking system with 0.78 ms temporal resolution in 200 ms exposure is experimentally demonstrated. Furthermore, it provides a new image format, Fourier-phase map, which has a compact data volume. The latent spatio-temporal information in one 2D image can be efficiently reconstructed at relatively low computation cost through the straightforward phase matching algorithm. Cooperated with scene-driven exposure as well as reasonable Fourier-phase prediction, one could acquire 4D data ( x , y , z , t ) of the moving objects, segment 3D motion based on temporal cues, and track targets in a complicated environment.
Adaptive milliseconds tracking and zooming optics based on a high-speed gaze controller and liquid lenses

Jiaqi Li,Lin Li,Lihui Wang,Lei Li,Shaoyong Li,Masatoshi Ishikawa

DOI: https://doi.org/10.1364/oe.512003

IF: 3.8

2024-01-09

Optics Express

Abstract:Jiaqi Li, Lin Li, Lihui Wang, Lei Li, Shaoyong Li, Masatoshi Ishikawa The high-speed gaze and high resolution are critical factors for actual monitoring systems. However, the conventional method cannot track ... [Opt. Express 32, 2257-2270 (2024)]

optics
Single Object Tracking With Fuzzy Least Squares Support Vector Machine

Shunli Zhang,Sicong Zhao,Yao Sui,Li Zhang

DOI: https://doi.org/10.1109/TIP.2015.2484068

IF: 10.6

2015-01-01

IEEE Transactions on Image Processing

Abstract:Single object tracking, in which a target is often initialized manually in the first frame and then is tracked and located automatically in the subsequent frames, is a hot topic in computer vision. The traditional tracking-by-detection framework, which often formulates tracking as a binary classification problem, has been widely applied and achieved great success in single object tracking. However...
Fast and high-precision tracking technology for image-based closed-loop cascaded control system with a Risley prism and fast steering mirror

Xia Huayang,yunxia Xia,Liangzhu Yuan,Piao Wen,Wenxue Zhang,Ke Ding,Fan Yue,Haotong Ma,Jinying Li,Huayang Xia,Yunxia Xia,Yue Fan

DOI: https://doi.org/10.1364/oe.511987

IF: 3.8

2024-02-28

Optics Express

Abstract:Huayang Xia, Yunxia Xia, Liangzhu Yuan, Piao Wen, Wenxue Zhang, Ke Ding, Yue Fan, Haotong Ma, Jinying Li The Risley prism's compact structure, dynamic responsiveness, and high tracking accuracy make it ideal for photoelectric image tracking. To ... [Opt. Express 32, 8555-8571 (2024)]

optics

Mask-based single-pixel tracking and imaging for moving objects

Fast and Sub-Pixel Precision Target Tracking Algorithm for Intelligent Dual-Resolution Camera.

Multi-modal 3D Human Tracking for Robots in Complex Environment with Siamese Point-Video Transformer

Image-free real-time 3-D tracking of a fast-moving object using dual-pixel detection

Single-pixel imaging of a translational object

Moving Object Tracking Research Based on Active Vision

Single-pixel Imaging of a Translational Object.

Prior-free 3D tracking of a fast-moving object at 6667 frames per second with single-pixel detectors

Tracking and fast imaging of a moving object via Fourier modulation

High-speed target tracking control system based on short-time rotational reflection imaging

Lexicality and Modality Effects on Evoked Potentials in a Memory-Scanning Task

Risley-prism-based tracking model for fast locating a target using imaging feedback

Non-line-of-sight imaging and tracking of moving objects based on deep learning

Gaze control system for tracking Quasi-1D high-speed moving object in complex background

An Active Multi-Object Ultrafast Tracking System with CNN-Based Hybrid Object Detection

Dynamic Object Tracking for Self-Driving Cars Using Monocular Camera and LIDAR.

Single-shot 3D Tracking Based on Polarization Multiplexed Fourier-phase Camera

Adaptive milliseconds tracking and zooming optics based on a high-speed gaze controller and liquid lenses

Single Object Tracking With Fuzzy Least Squares Support Vector Machine

Fast and high-precision tracking technology for image-based closed-loop cascaded control system with a Risley prism and fast steering mirror