Real-Time 3D Tracking of Multi-Particle in the Wide-Field Illumination Based on Deep Learning

Xiao Luo,Jie Zhang,Handong Tan,Jiahao Jiang,Junda Li,Weijia Wen

DOI: https://doi.org/10.3390/s24082583

IF: 3.9

2024-04-19

Sensors

Abstract:In diverse realms of research, such as holographic optical tweezer mechanical measurements, colloidal particle motion state examinations, cell tracking, and drug delivery, the localization and analysis of particle motion command paramount significance. Algorithms ranging from conventional numerical methods to advanced deep-learning networks mark substantial strides in the sphere of particle orientation analysis. However, the need for datasets has hindered the application of deep learning in particle tracking. In this work, we elucidated an efficacious methodology pivoted toward generating synthetic datasets conducive to this domain that resonates with robustness and precision when applied to real-world data of tracking 3D particles. We developed a 3D real-time particle positioning network based on the CenterNet network. After conducting experiments, our network has achieved a horizontal positioning error of 0.0478 μm and a z-axis positioning error of 0.1990 μm. It shows the capability to handle real-time tracking of particles, diverse in dimensions, near the focal plane with high precision. In addition, we have rendered all datasets cultivated during this investigation accessible.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

What problem does this paper attempt to address?

The paper attempts to address the problem of real-time three-dimensional tracking of multiple particles under wide-field illumination conditions. Specifically, the researchers focus on the importance of particle localization and analysis in various research fields, such as holographic optical tweezers mechanics measurement, colloidal particle motion state detection, cell tracking, and drug delivery. Despite significant progress made by traditional numerical methods and advanced deep learning networks in particle orientation analysis, the application of deep learning in particle tracking is still limited by the demand for datasets. To this end, this paper proposes an effective deep learning-based method for generating synthetic datasets suitable for this field, and this method demonstrates robustness and accuracy when applied to real data. The researchers developed a 3D real-time particle localization network based on the CenterNet network. Experimental results show that this network can handle high-precision real-time tracking of particles of different sizes near the focal plane, with a horizontal localization error of 0.0478 µm and a Z-axis localization error of 0.1990 µm. Additionally, the researchers have made all datasets generated during the research publicly available.