Abstract:Speckle projection profilometry (SPP), which is highly suited for dynamic 3D acquisition, can build the global correspondences between stereo images by projecting a single random speckle pattern. But SPP suffers from the low matching accuracy of traditional stereo matching algorithms which limits its 3D measurement quality and precludes the recovery of the fine details of complex surfaces. For enhancing the matching precision of SPP, in this paper, we propose an end-to-end speckle matching network for 3D imaging. The proposed network first leveraged a multi-scale residual subnetwork to synchronously extract feature maps of stereo speckle images from two perspectives. Considering that the cost filtering based on 3D convolution is computationally costly, the 4D cost volume with a quarter of the original resolution is established and implemented cost filtering to achieve higher stereo matching performance with lower computational overhead. In addition, for the dataset of SPP built for supervised deep learning, the label of the sample data only has valid values in the foreground. Therefore, in our work, a simple and fast saliency detection network is integrated into our end-to-end network, which takes as input the features computed from the shared feature extraction subnetwork of the stereo matching network and produces first a low-resolution invalidation mask. The mask is then upsampled and refined with multi-scale multi-level residual layers to generate the final full-resolution mask. This allows our stereo matching network to avoid predicting the invalid pixels in the disparity maps, such as occlusions, backgrounds, thereby implicitly improving the disparity accuracy for valid pixels. The experiment results demonstrated that the proposed method can achieve fast and absolute 3D shape measurement with an accuracy of about 100um through a single speckle pattern.

Learning-based absolute 3D shape measurement based on single fringe phase retrieval and speckle correlation

Composite Deep Learning Framework for Absolute 3D Shape Measurement Based on Single Fringe Phase Retrieval and Speckle Correlation

Learning-based Absolute 3D Shape Measurement Based on Single Fringe Phase Retrieval and Speckle Correlation

Deep-learning-enabled geometric constraints and phase unwrapping for single-shot absolute 3D shape measurement

Single-shot 3D shape measurement with spatial frequency multiplexing using deep learning

High-speed 3D shape measurement using the optimized composite fringe patterns and stereo-assisted structured light system

Deep-learning-enabled Dual-Frequency Composite Fringe Projection Profilometry for Single-Shot Absolute 3D Shape Measurement

Dynamic 3-D measurement based on fringe-to-fringe transformation using deep learning

High-speed 3D shape measurement from noisy fringe images using deep learning

Three-dimensional Shape Measurement Based on Speckle-Embedded Fringe Patterns and Wrapped Phase-to-height Lookup Table

Absolute Phase Retrieval for a Single-Shot Fringe Projection Profilometry Based on Deep Learning

A Multi-Code 3D Measurement Technique Based on Deep Learning

Untrained Deep Learning-Based Phase Retrieval for Fringe Projection Profilometry

Deep Learning-Based 3D Measurements with Near-Infrared Fringe Projection

3D Shape Measurement Using Speckle-Embedded Fringes and Lookup Table

Unambiguous 3D Measurement from Speckle-Embedded Fringe.

Real-time 3D Shape Measurement of Dynamic Scenes Using Fringe Projection Profilometry: Lightweight NAS-optimized Dual Frequency Deep Learning Approach.

High-speed 3D Shape Measurement with the Multi-View System Using Deep Learning

Single-Shot Multi-Frequency 3D Shape Measurement for Discontinuous Surface Object Based on Deep Learning

End-to-end Single-Shot Composite Fringe Projection Profilometry Based on Deep Learning

Single-shot 3D Shape Measurement Using an End-to-end Stereo Matching Network for Speckle Projection Profilometry.