Deep Learning-Based Three-dimensional Spatial Distribution Reconstruction for Light Field Micro-Particle Image Velocimetry with Convolutional Neural Network
Shen Shiyu,Li Jian,Gu Mengtao,Zhang Biao,Xu Chuanlong
DOI: https://doi.org/10.3788/AOS230958
2023-01-01
Acta Optica Sinica
Abstract:Objective Light field micro-particle image velocimetry (LF-mu PIV) can measure the three-dimensional (3D) velocity field of microflow by a single light field camera. The 3D spatial distribution reconstruction of tracer particles is significant in LF-mu PIV. Model-based approaches, including refocusing technology and deconvolution method, are conventionally adopted for the reconstruction. However, the refocusing technology ignores the diffraction effect of the microscope and simplifies the microlens as a pinhole, resulting in low lateral resolution and axial positioning accuracy of the reconstructed tracer particle. Although the deconvolution method improves the lateral resolution based on wave optics theory, the axial resolution is still low due to the limited light-receiving angle of the imaging system. Additionally, the laterally shift-variant point spread function lowers the reconstruction efficiency of the deconvolution method. To this end, the data-driven approach, e.g., deep learning technique, is proposed to achieve the volumetric reconstruction of the tracer particle distribution. Generally, additional high-resolution 3-D imaging devices such as confocal and selective-plane illumination microscopes are required to establish the 'particle spatial distribution-light field image' dataset. However, they are costly and difficult to implement for the dynamic flow process due to their extremely low temporal resolution. We propose a deep learning-based 3D spatial distribution reconstruction for LF-mu PIV with convolutional neural networks to rapidly reconstruct particle distribution with high resolution. Methods With the imaging model of tracer particles in a light field microscope based on the wave optics theory, the light field images are formed through numerical simulations based on the actual luminous characteristics of the particles to efficiently establish the "particle spatial distribution-light field image" dataset. Afterward, the sub-aperture images are extracted from the light field image to acquire angle information since the 2D light field image contains 3D spatial distribution information of tracer particles. The sub-aperture images are employed as multi-channel input for feature extraction to achieve the mapping between the light field images and the 3D spatial distribution of tracer particles with a deep learning model based on convolutional neural networks. As a result, a prediction model for reconstructing the particle spatial distribution is obtained. Further, the reconstruction quality and resolution, particle extraction rate, reconstruction efficiency, and anti-noise performance of the prediction model are evaluated in a test set. Finally, the 3D particle distribution and the velocity field in a horizontal microchannel laminar flow are experimentally measured to verify the practicability of the proposed method. Results and Discussions In the simulation, the axial full widths at half maximum of reconstructed particles for the proposed method and deconvolution method are 2. 34 mu m and 11. 30 mu m respectively, which indicates that the proposed deep learning method improves the axial resolution by 79. 3% (Table 2). As a result, within the particle concentration range of 0. 3 to 1. 2 (represent particle concentration by the number of particles corresponding to each microlens), the proposed method always has a higher reconstruction quality than the deconvolution method (Fig. 11). In terms of reconstruction efficiency, the proposed method has significant improvements compared with the deconvolution method. Specifically, the proposed method only takes 0. 243 s to achieve the 3D spatial distribution reconstruction of tracer particles, while the deconvolution method takes 31133 s (Table 3). Notably, although the reconstruction quality of the proposed method would be degraded due to the noise, it is still better than the deconvolution method, showing that the proposed method has sound anti-noise performance (Fig. 15). In the experimental evaluations, the reconstructed particle distributions of the proposed method and deconvolution method are basically consistent despite the differences in particle intensity (Fig. 17). The experimental axial full widths at half maximum of reconstructed particles for the proposed method and deconvolution method are 2. 82 mu m and 13. 20 mu m respectively, which are similar with the simulated results (Fig. 17). Meanwhile, the measured velocity distribution consistent with the theoretical value verifies the feasibility of the proposed method for LF-mu PIV (Fig. 19). Conclusions We propose a deep learning-based 3D spatial distribution reconstruction for LF-mu PIV with convolutional neural networks to rapidly reconstruct particle distribution with high resolution. According to the imaging model of the light field microscope, light field images are numerically formed based on the actual luminous particle characteristics to efficiently construct the 'particle spatial distribution-light field image' dataset. Afterward, a deep learning model based on convolutional neural networks is built and trained through the dataset to obtain a prediction model for reconstructing the spatial particle distribution. The reconstruction performance of the prediction model is evaluated in a test set. Finally, the 3D particle distribution and the velocity field in a horizontal microchannel laminar flow are experimentally measured with the proposed method. Results show that the proposed method improves axial resolution by 79. 3% compared with the deconvolution method. The reconstruction time for a single light field image through the proposed method is only 0. 243 seconds, which meets the real-time measurement demands. The measured velocity distribution consistent with the theoretical value further verifies the feasibility of the proposed method for LF-mu PIV.