High dynamic range three-dimensional measurement using fringe projection technique

shijie feng,qian chen,chao zuo,yuzhen zhang

DOI: https://doi.org/10.1364/ISA.2014.ITu3C.4

2014-01-01

Abstract:This paper presents an approach to high dynamic range shape measurement for scenario of largely varying range of surface reflectivity, e.g. simultaneously containing very dark and bright surfaces.

What problem does this paper attempt to address?

High dynamic range 3D measurements with fringe projection profilometry: a review

Shijie Feng,Liang Zhang,Chao Zuo,Tianyang Tao,Qian Chen,Guohua Gu

DOI: https://doi.org/10.1088/1361-6501/aae4fb

IF: 2.398

2018-11-05

Measurement Science and Technology

Abstract:In optical metrology, fringe projection is considered to be one of the most reliable techniques for recovering the shape of objects. For this technique, however, it is challenging to measure objects with a large variation in surface reflectivity, e.g. a scenario containing both dark and bright objects. Researchers have thus developed various approaches to high dynamic range (HDR) three-dimensional (3D) measurements over the years. In this paper, we present an overview of these techniques, as well as a new and definitive classification for them. We implement a set of representative techniques to measure objects with different characteristics of reflectance and discuss the advantages and constraints of the techniques according to the comparative results. Moreover, challenging problems and future research directions are discussed to advance HDR 3D measurement techniques.

engineering, multidisciplinary,instruments & instrumentation
Surface Measurement Methods Based on Single and Multi-Frequency Fringes and Its Applications in Reverse Engineering

Yong Jian Zhu,Jing Xin Na,Jian Feng Sun,Shao Hui Yin

DOI: https://doi.org/10.4028/www.scientific.net/amm.148-149.460

2011-01-01

Applied Mechanics and Materials

Abstract:The 3D surface measurement methods are described based on the single and multi-frame fringe projection principles, respectively. The key algorithm  phase reconstructing (PR) is conducted by means of two different concepts; one is the single-frequency spatial phase reconstructing method and the other is the multi-frequency temporal one. On the side of spatial PR, a few typical methods are carried out, among which the most robust one could be achieved. The described spatial methods are implemented under the guide of the suggested modulation gradient variance (MGV) map. Moreover, the network programming (NP) method is also conducted to compare with them. The tested results show that the weighted least square (WLS) is the most robust one among the spatial PR methods. At the same time, the temporal PR method is introduced based on the multi-frequency fringes, and the result proves that it features the better noise immunity and robustness than spatial ones, but needs many different frequencies of fringes that would be more time-consuming. Lastly, the surfaces are reconstructed in reverse engineering (RE) software.
3D Measurement System Based on Computer-Generated Gratings

Yongjian Zhu,Weiqing Pan,Yanliang Luo

DOI: https://doi.org/10.1117/12.885280

2010-01-01

Abstract:A new kind of 3D measurement system has been developed to achieve the 3D profile of complex object. The principle of measurement system is based on the triangular measurement of digital fringe projection, and the fringes are fully generated from computer. Thus the computer-generated four fringes form the data source of phase-shifting 3D profilometry. The hardware of system includes the computer, video camera, projector, image grabber, and VGA board with two ports (one port links to the screen, another to the projector). The software of system consists of grating projection module, image grabbing module, phase reconstructing module and 3D display module. A software-based synchronizing method between grating projection and image capture is proposed. As for the nonlinear error of captured fringes, a compensating method is introduced based on the pixel-to-pixel gray correction. At the same time, a least square phase unwrapping is used to solve the problem of phase reconstruction by using the combination of Log Modulation Amplitude and Phase Derivative Variance (LMAPDV) as weight. The system adopts an algorithm from Matlab Tool Box for camera calibration. The 3D measurement system has an accuracy of 0.05mm. The execution time of system is 3 similar to 5s for one-time measurement.
High dynamic range 3D measurement based on polarization and multispectrum co-modulation

Haorui Huang,Fuqian Li,Chenglin Zuo,Yajun Wang

DOI: https://doi.org/10.1364/OE.506216

2023-12-04

Abstract:Three-dimensional (3D) shape measurement serves an important role in many areas, and fringe projection profilometry (FPP) is a widely used 3D measurement technique due to its non-physical contact and high speed. The real measurement scenarios are often mixtures of specular and diffuse reflections, causing overexposed and underexposed areas to co-exist. Currently, utilizing FPP to simultaneously measure overexposed and underexposed areas remains a challenge. To solve this problem, we propose a mixed reflection model and what we believe to be a novel high dynamic range (HDR) 3D measurement method based on polarization and multispectrum co-modulation. In mixed reflection, the fringe images captured by the polarized color camera can be modulated to different intensities between different channels due to the co-modulation effect. By synthesizing all sub-images, high-modulation fringe images are formed and simultaneous reconstruction of overexposed and underexposed surfaces is finally achieved. Compared to conventional methods, the proposed method is more effective for measuring complex reflection situations, especially when objects with specular and diffuse surfaces simultaneously exist in the scene. And what we believe to be a novel no-registration-error calibration framework for multi-channel cameras has also been proposed, which both acquires a significant amount of information in the region with HDR problems and avoids the registration error due to the physical distances between different channels. Experiments were conducted to verify the effectiveness of the proposed method.
Efficient 3D measurement of a HDR surface based on adaptive fringe projection

Jialing Hu,Jiangping Zhu,Pei Zhou

DOI: https://doi.org/10.1364/AO.470064

IF: 1.9

2022-10-20

Applied Optics

Abstract:Jialing Hu, Jiangping Zhu, Pei Zhou 3D measurement methods of a high-dynamic-range (HDR) surface based on adaptive fringe projection have aroused extensive research interest. ... [Appl. Opt. 61, 9028-9036 (2022)]

optics
High Dynamic Range Structured Light 3-D Measurement Based on Region Adaptive Fringe Brightness

Yichen Fu,Junfeng Fan,Fengshui Jing,Min Tan

DOI: https://doi.org/10.1109/tie.2023.3303655

IF: 7.7

2023-01-01

IEEE Transactions on Industrial Electronics

Abstract:Fringe projection profilometry is one of the main methods for active 3-D measurement, but practical problems, such as how to obtain a complete point cloud by overcoming the effects of high surface reflectivity of objects, remain to be solved. So, in this article, we propose a set of hardware devices and region adaptive structured light algorithms. First, we introduce the surface structured light measurement system adopted in this article. Second, based on the accurate optical modeling of the measurement scene, the principles for generating the adaptive optimal projection brightness are derived in detail and the process is accelerated by exploiting the speed of neural network. Meanwhile, we creatively use the chain code combined with the M-estimator sample consensus method to find the homography from the saturated region of the camera plane to the corresponding region of the projector plane for generating fringe images with adaptive brightness. Furthermore, we chose the more robust line-shifting code and binarization method for subpixel 3-D reconstruction. Experiments have shown that our work is efficient, superior, and accurate in measuring objects with high reflective surface properties.

automation & control systems,engineering, electrical & electronic,instruments & instrumentation
An adaptive fringe projection method for 3D measurement with high-reflective surfaces

Haisen Yuan,Yu Li,Jing Zhao,Ling Zhang,Wenjie Li,Yang Huang,Xingyu Gao,Qiadong Xie

DOI: https://doi.org/10.1016/j.optlastec.2023.110062

IF: 4.939

2023-10-18

Optics & Laser Technology

Abstract:Fringe projection profilometry (FPP) has the advantages of non-contact, quick and high accuracy, which is popular in the fields of industrial inspection and measurement. However, the coded fringe patterns may be failure suffered from the high reflectivity of the measured surfaces. Aiming at this problem, an adaptive fringe projection method is proposed in this paper by the best projection intensity strategy and the coordinate mapping establishment between camera and projection regions. Combining the local camera response function, the dichotomizing search is used to confirm the best projection intensity of overexposed regions by pixel-wise. The unwrapping phase map of background is performed to establish the coordinate mapping relationship of measured overexposed regions between the camera and projection images, which is robust and convenient. The mapping errors caused by inaccurate phase information in the overexposed regions are effectively avoided, and the fringe projection patterns are adjusted adaptively, without additional equipment and low-expose patterns. Both theory and experiments have verified the feasibility and superiority of the proposed method.

optics,physics, applied
High-Fidelity Surface Reconstruction Using Complementary Colorimetry-Encoded Fringe Projection Profilometry

Yubo Ni,Ying Wang,Yajing Bai,Zhaozong Meng,Nan Gao,Hongwei Zhao,Guofeng Zheng,Jing Xu,Zonghua Zhang

DOI: https://doi.org/10.1364/oe.533068

IF: 3.8

2024-01-01

Optics Express

Abstract:To solve the challenging issue of uneven reflectance of high-fidelity surface, this paper proposes a three-dimensional (3D) shape measurement method based on surface adaptation complementary colorimetry-encoded fringe projection profilometry. A complete complementary hue extraction method simulates the retina stimulation to establish the complementary conversion relationship between the projected and collected colors of the measurement system. On this basis, a surface reflectance adaptation fringe pattern encoding strategy is proposed to measure the highfidelity surfaces. The experiments demonstrate that the proposed method is capable of efficiently balancing and adjusting the reflected light intensity of each fidelity surface, thereby suppressing the ambiguity reconstruction texture. The established complementary colorimetry relationship also improves the light wavelength discrimination, saturation avoidance, measurement accuracy and efficiency.
3D shape measurement method for high-reflective surface based on a color camera

Zhangying Wang,Kui Li,Nan Gao,Zhaozong Meng,Zonghua Zhang

DOI: https://doi.org/10.1364/oe.542643

IF: 3.8

2024-10-31

Optics Express

Abstract:Zhangying Wang, Kui Li, Nan Gao, Zhaozong Meng, Zonghua Zhang Fringe projection profilometry (FPP) has been widely utilized in many fields due to its non-contact, high accuracy, high resolution and ... [Opt. Express 32, 41603-41618 (2024)]

optics
3D shape measurement method for multi-reflective scenes based on accurate pixel-by-pixel phase-error adjustment strategy

Luyuan Feng,Jiehu Kang,Leiwen Yuan,Hongtong Li,Yifei Chen,Zhen Zhang,Zefeng Sun,Jian Liang,Bin Wu

DOI: https://doi.org/10.1016/j.optlastec.2024.110661

IF: 4.939

2024-02-04

Optics & Laser Technology

Abstract:Fringe projection profilometry (FPP) has been widely applied to three-dimensional (3D) shape measurement with the advantages of non-contact and high accuracy. However, measuring the scenes of multi-reflective variations (MRVs) is still a tricky task for conventional FPP because the dark-field and saturation could lead to phase errors and reconstruction errors. In this paper, a novel high dynamic range (HDR) technique based on accurate pixel-by-pixel phase-error adjustment strategy is proposed, which is mainly focus on both accuracy and quality of the reconstruction. A reflection model and an error transformation model are built to analyze the disturbances induced by MRVs in the captured fringe patterns. By setting a proper phase error threshold, the exposure time range of each pixel can be calculated. Finally, a greedy algorithm is introduced to globally minimize the number of exposures. Experiments results have verified that our proposed method can improve both the measurement accuracy and accurate point cloud integrality (APCI) compared with conventional FPP and several HDR methods, making it suitable to be further applied in the HDR 3D shape measurement.

optics,physics, applied
High Dynamic Range Scanning Technique

Song Zhang,Shing-Tung Yau

DOI: https://doi.org/10.1117/12.791265

2008-01-01

Abstract:Measuring objects with high surface reflectivity variations (i.e., high dynamic range) is challenging for any optical method. This paper addresses a high dynamic range scanning (HDRS) technique that can measure this type of objects. It takes advantage of one merit of a phase-shifting algorithm: pixel-by-pixel phase retrieval. For each measurement, multiple shots of fringe images with different exposures are taken. And a sequence of fringe images with different overall brightness are captured: the brightest fringe images have good fringe quality for darker areas although the brighter areas may be saturated; while the darkest fringe images have good fringe quality in brighter areas although the fringes in the darker areas may be invisible. The sequence of fringe images is arranged from brighter to darker, i.e., from higher exposure to lower exposure. The final fringe images, used for phase retrieval, are produced pixel-by-pixel by choosing the brightest but the unsaturated corresponding pixel from one shot. A phase-shifting algorithm is employed to compute the phase, which can be further converted to coordinates. Our experiments demonstrate that the proposed technique can successfully measure objects with high dynamic range of surface properties.
High-reflectivity surface measurement in structured-light technique by using a transparent screen

Jianbin Cao,Chengcheng Li,Chen Li,Xu Zhang,Dawei Tu

DOI: https://doi.org/10.1016/j.measurement.2022.111273

IF: 5.6

2022-04-01

Measurement

Abstract:Fringe projection profilometry (FPP) has been widely applied to measure objects, but for measurement of the high-reflectivity surfaces is still a tricky problem. In this paper, a new high-dynamic-range (HDR) technique by using a transparent screen as an optical mask for the camera is proposed. Specially, a transparent screen is placed in front of the camera, and each camera pixel’s intensity can be precisely controlled by adjusting its corresponding screen pixel’s intensity. Experiments has verified the effectiveness of our method: the dynamic range of the 4-step phase-shift method is increased from 48.13 dB to 71.83 dB , and a fewer number of projection patterns compared with the other techniques. The significance of this study is that our method can be easily combined with the other HDR technologies to obtain a greater dynamic range for the structured light technique.

engineering, multidisciplinary,instruments & instrumentation
Microscopic 3D measurement of shiny surfaces based on a multi-frequency phase-shifting scheme

Yan Hu,Qian Chen,Yichao Liang,Shijie Feng,Tianyang Tao,Chao Zuo

DOI: https://doi.org/10.1016/j.optlaseng.2019.05.019

2018-12-31

Abstract:Microscopic fringe projection profilometry is a powerful 3D measurement technique with a theoretical measurement accuracy better than one micron provided that the measured targets can be imaged with good fringe visibility. However, practically, the 3D shape of the measured surface can hardly be fully reconstructed due to the defocus of the dense fringes and complex surface reflexivity characteristics, which lead to low fringe quality and intensity saturation. To address this problem, we propose to calculate phases of these highlighted areas from a subset of the fringe sequence which is not subjected to the intensity saturation. By using the proposed multi-frequency phase-shifting scheme, the integrity of the 3D surface reconstruction can be significantly improved. The ultimate phase maps obtained from unsaturated intensities are used to achieve high-accuracy 3D recovering of shiny surfaces based on a phase stereo matching method. Experimental results on different metal surfaces show that our approach is able to retrieve the complete morphology of shiny surfaces with high accuracy and fidelity.

Image and Video Processing,Optics
Dynamic three-dimensional sensing for specular surface with monoscopic fringe reflectometry.

Lei Huang,Chi Seng Ng,Anand Krishna Asundi

DOI: https://doi.org/10.1364/OE.19.012809

IF: 3.8

2011-01-01

Optics Express

Abstract:Dynamic full-field three-dimensional sensing of specular reflective surfaces can be conveniently implemented with fringe reflection technique. A monoscopic fringe reflectometric system can be adopted as a simple measuring setup. With the assistance of the windowed Fourier ridges method as an advanced fringe demodulation technique, only one cross grating is needed to reconstruct the three-dimensional surface shape changes. A suitable calibration enables determination of the actual three-dimensional surface profile. Experimental results of water wave variations are shown to demonstrate the feasibility of the proposed approach. (C)2011 Optical Society of America
An investigation of high dynamic range imaging of highly reflective surfaces based on deep learning 3D fringe projection

wei gao,Shixiang Wang

DOI: https://doi.org/10.1117/12.3009055

2023-12-29

Abstract:Three-dimensional reconstruction of highly reflective surfaces has always been an important and difficult problem in structured light technology. In this paper, a novel method by combining optimal exposure with the adjustment of the intensity of projection fringes by deep learning is proposed a single frame method is used to extract the overexposed region in the optimal exposure time series for composite projection. By using the statistical histogram of the number of overexposed pixels of the measured object under different projection conditions, and combining with the proportion of pixels in the image that are not exposed, the optimal exposure time series is calculated and the overexposed area is automatically extracted. The single-frame composite high dynamic image is obtained by adjusting the projection intensity through absolute coordinates, which improves the single-frame high dynamic range technique for highly reflective surfaces. Several simulations and experiments were conducted. The experimental results show that the proposed method can effectively improve the 3D reconstruction accuracy of the highly reflective surface, and the number of effective pixels in the point cloud can reach more than 98%.

Engineering,Physics,Computer Science
A 3D Shape Measurement Method for High-reflective Surface Based on Dual-view Multi-intensity Projection

Peng Zhou,Hengyu Wang,Yao Wang,Chuanwei Yao,Bin Lin

DOI: https://doi.org/10.1088/1361-6501/accbe0

IF: 2.398

2023-04-13

Measurement Science and Technology

Abstract:Phase shifting profilometry (PSP) has been commonly used in three-dimensional shape measurement with the advantages of high-precision and non-contact. However, it is still challenging to measure high-reflective surface because image saturation will lead to absolute phase errors and reconstruction errors. In this paper, a dual-view multi-intensity projection method was proposed. Compared with the single-view method, the proposed method can reconstruct more points at each projection intensity especially for pixels around the specular angle and reduce the number of projections to reduce the time consumption. First, we established the dual-view structured light system consisting of two monocular systems that share the same projector. Subsequently, a dual-view saturated pixel judging method was proposed that enables the reconstruction results under two views to be combined without duplicate points. The multi-intensity projection method was adopted by reducing the input projection intensity step by step and reconstructing the remaining pixels around the specular angle. Finally, the reconstruction result can be obtained by stitching point clouds at each projection intensity. Experiments verified that the proposed method could improve the integrity of reconstructed point clouds and measurement efficiency.

engineering, multidisciplinary,instruments & instrumentation
High-Dynamic-Range 3-D Shape Measurement With Adaptive Speckle Projection Through Segmentation-Based Mapping

Pei Zhou,Yue Cheng,Jiangping Zhu,Jialing Hu

DOI: https://doi.org/10.1109/tim.2022.3218110

IF: 5.6

2023-01-20

IEEE Transactions on Instrumentation and Measurement

Abstract:The 3-D shape measurement of high-dynamic-range (HDR) scenes based on structured light projection is challenging because the conventional projection without considering the reflection distribution of the tested surface usually causes over-exposure and over-dark areas simultaneously in the captured images, which will lead to the failure or low accuracy of 3-D reconstruction. In this article, we propose a novel HDR 3-D surface measurement method based on adaptive speckle projection with fully considering the reflection distribution. An efficient segmentation-based mapping strategy is established to speed up the generation of adaptive speckle patterns, which segments the estimated mask with optimal projection intensity into multiple subregions by the eight-connected standard to raise mapping accuracy and project gray-coded and sinusoidal fringe patterns to build the pixelwise mapping relationship between binocular cameras and the projector for each subregion, respectively. Then the pixelwise intensity of projected speckle patterns can be accurately adjusted according to optimal intensity. The generated adaptive speckle pattern is used to scan the tested object, and high signal-to-noise-ratio (SNR) speckle images can be obtained for accurate and complete 3-D measurement. Experimental results verify that the point cloud produced by our method has obvious advantages compared with the traditional methods in terms of measurement accuracy, whose coverage is as large as 92.26% and the average distance (AD) relative to ground truth (GT) is as low as 0.128 mm. Meanwhile, the number of projected patterns for mapping is decreased by a third compared with the widely used mapping strategy.

engineering, electrical & electronic,instruments & instrumentation
Pixel-wise Phase Map Fusion Technique for High Dynamic Range 3-D Shape Measurement

Yifei Chen,Jiehu Kang,Luyuan Feng,Leiwen Yuan,Bin Wu

DOI: https://doi.org/10.1088/1361-6501/ad7da6

IF: 2.398

2024-09-21

Measurement Science and Technology

Abstract:Digital fringe projection technologies have been extensively applied in 3-D shape measurements. However, it is still limited to achieve accurate 3-D reconstruction of high reflective surfaces. To circumvent this problem, we propose a complete framework to realize pixel-by-pixel phase post-compensation of overexposed points, including the construction of hardware system and the guidelines of algorithm design. First, a virtual binocular vision system is established. Second, based on the fundamentals of projective and spatial transformation, three robust algorithms with low computational complexity are devised to generate a fused phase map that overcomes the information loss due to overexposure. Furthermore, experiments are conducted to evaluate the performance of the proposed method. Qualitative experiment results show that the presented technique achieve a similar surface coverage compared with the other two mainstream methods, but is superior to them in terms of measurement efficiency (13 images vs. 144 images, 13 images vs. 32 images). Quantitative experimentation further verifies the feasibility of our method in precise 3-D measurement, with sphere diameter errors of 0.0367mm and 0.0361mm, spherical centroid distance error of 0.0269mm, and plane distance error of 0.0186mm.

engineering, multidisciplinary,instruments & instrumentation
High-speed high dynamic range 3D shape measurement based on deep learning

Liang Zhang,Qian Chen,Chao Zuo,Shijie Feng

DOI: https://doi.org/10.1016/j.optlaseng.2020.106245

IF: 5.666

2020-11-01

Optics and Lasers in Engineering

Abstract:<p>For many three-dimensional (3D) measurement techniques based on fringe projection profilometry (FPP), measuring the objects with a large variation range of surface reflectivity is always a very tricky problem due to the limited dynamic range of camera. Many high dynamic range (HDR) 3D measurement methods are developed for static scenes, which are fragile for dynamic objects. In this paper, we address the problem of phase information loss in HDR scenes, in order to enable 3D reconstruction from saturated or dark images by deep learning. By using a specifically designed convolutional neural network (CNN), we can accurately extract phase information in both the low signal-to-noise ratio (SNR) and saturation situations after proper training. Experimental results demonstrate the success of our network in 3D reconstruction for both static and dynamic HDR objects. Our method can improve the dynamic range of three-step phase-shifting by a factor of 4.8 without any additional projected images or hardware adjustment during measurement. And the final 3D measurement speed of our method is about 13.89 Hz (off-line).</p>

optics
Rapid 3D measurement of colour objects based on three-channel sinusoidal fringe projection

Bao Wei,Fu Yanjun,Zhong Kejun,Ma Baiheng,Yan Zhanjun

DOI: https://doi.org/10.1080/09500340.2022.2090627

IF: 1.3

2022-06-22

Journal of Modern Optics

Abstract:A rapid 3D measurement method based on three-channel projection is proposed in this paper. Project a composite color fringe map and an all-white map to the object to be measured. Firstly, crosstalk correction is performed on the composite color fringe pattern collected by the camera then the reflectivity of fringe image is corrected by all-white image. Then, the background normalization principle is used to solve the spectrum aliasing problem when the Fourier transform obtains a truncated phase. Finally, the three-frequency heterodyne method is used for phase unwrapping to obtain the continuous phase and 3D shape. Compared with the traditional color fringe projection method, this method not only eliminates the spectrum aliasing problem but also solves the uneven fringe reflectivity problem when measuring color objects using an all-white map. This method only requires two images to achieve measurement of 3D color dynamic objects.

optics

High dynamic range three-dimensional measurement using fringe projection technique

High dynamic range 3D measurements with fringe projection profilometry: a review

Surface Measurement Methods Based on Single and Multi-Frequency Fringes and Its Applications in Reverse Engineering

3D Measurement System Based on Computer-Generated Gratings

High dynamic range 3D measurement based on polarization and multispectrum co-modulation

Efficient 3D measurement of a HDR surface based on adaptive fringe projection

High Dynamic Range Structured Light 3-D Measurement Based on Region Adaptive Fringe Brightness

An adaptive fringe projection method for 3D measurement with high-reflective surfaces

High-Fidelity Surface Reconstruction Using Complementary Colorimetry-Encoded Fringe Projection Profilometry

3D shape measurement method for high-reflective surface based on a color camera

3D shape measurement method for multi-reflective scenes based on accurate pixel-by-pixel phase-error adjustment strategy

High Dynamic Range Scanning Technique

High-reflectivity surface measurement in structured-light technique by using a transparent screen

Microscopic 3D measurement of shiny surfaces based on a multi-frequency phase-shifting scheme

Dynamic three-dimensional sensing for specular surface with monoscopic fringe reflectometry.

An investigation of high dynamic range imaging of highly reflective surfaces based on deep learning 3D fringe projection

A 3D Shape Measurement Method for High-reflective Surface Based on Dual-view Multi-intensity Projection

High-Dynamic-Range 3-D Shape Measurement With Adaptive Speckle Projection Through Segmentation-Based Mapping

Pixel-wise Phase Map Fusion Technique for High Dynamic Range 3-D Shape Measurement

High-speed high dynamic range 3D shape measurement based on deep learning

Rapid 3D measurement of colour objects based on three-channel sinusoidal fringe projection