Robust structured light with efficient redundant codes

Zhanghao Sun,Xinxin Zuo,Dong Huo,Yu Zhang,Yiming Qian,Jian Wang

DOI: https://doi.org/10.1364/oe.533499

IF: 3.8

2024-09-05

Optics Express

Abstract:Zhanghao Sun, Xinxin Zuo, Dong Huo, Yu Zhang, Yiming Qian, Jian Wang Structured light (SL) systems acquire high-fidelity 3D geometry with active illumination projection. Conventional systems exhibit ... [Opt. Express 32, 33507-33520 (2024)]

optics

What problem does this paper attempt to address?

Structured Light with Redundancy Codes

Zhanghao Sun,Yu Zhang,Yicheng Wu,Dong Huo,Yiming Qian,Jian Wang

DOI: https://doi.org/10.48550/arXiv.2206.09243

2022-06-18

Computer Vision and Pattern Recognition

Abstract:Structured light (SL) systems acquire high-fidelity 3D geometry with active illumination projection. Conventional systems exhibit challenges when working in environments with strong ambient illumination, global illumination and cross-device interference. This paper proposes a general-purposed technique to improve the robustness of SL by projecting redundant optical signals in addition to the native SL patterns. In this way, projected signals become more distinguishable from errors. Thus the geometry information can be more easily recovered using simple signal processing and the ``coding gain" in performance is obtained. We propose three applications using our redundancy codes: (1) Self error-correction for SL imaging under strong ambient light, (2) Error detection for adaptive reconstruction under global illumination, and (3) Interference filtering with device-specific projection sequence encoding, especially for event camera-based SL and light curtain devices. We systematically analyze the design rules and signal processing algorithms in these applications. Corresponding hardware prototypes are built for evaluations on real-world complex scenes. Experimental results on the synthetic and real data demonstrate the significant performance improvements in SL systems with our redundancy codes.
Target Enhanced 3D Reconstruction Based on Polarization-Coded Structured Light.

Xiao Huang,Jian Bai,Kaiwei Wang,Qun Liu,Yujie Luo,Kailun Yang,Xianjing Zhang

DOI: https://doi.org/10.1364/oe.25.001173

IF: 3.8

2017-01-01

Optics Express

Abstract:Structured light is a prevailing and reliable active approach of 3D object reconstruction. But complex ambience is undesirable in the measurement because it could cause severe noise and increase computing overhead. In this paper, we propose a structured light coded by spatially-distributed polarization state of the illuminating patterns. The proposed structured light has the advantage of enhancing target in 3D reconstruction by polarization cues. Specifically, this method can estimate the degree of linear polarization (DOLP) in the scene, distinguish target by DOLP and selectively reconstruct it. The coding strategy and the corresponding polarimetric principle are presented and verified by experimental results. As our approach takes advantage of the intrinsic properties of liquid crystal display (LCD) projector and requires no rotation of polarizer, it is effective and efficient for practical applications.
Robust photometric stereo using structural light sources

Tian-Qi Han,Yue Cheng,Hui-Liang Shen,Xin Du

DOI: https://doi.org/10.1117/1.JEI.23.3.033004

IF: 0.829

2014-01-01

Journal of Electronic Imaging

Abstract:We propose a robust photometric stereo method by using structural arrangement of light sources. In the arrangement, light sources are positioned on a planar grid and form a set of collinear combinations. The shadow pixels are detected by adaptive thresholding. The specular highlight and diffuse pixels are distinguished according to their intensity deviations of the collinear combinations, thanks to the special arrangement of light sources. The highlight detection problem is cast as a pattern classification problem and is solved using support vector machine classifiers. Considering the possible misclassification of highlight pixels, the l(1) regularization is further employed in normal map estimation. Experimental results on both synthetic and real-world scenes verify that the proposed method can robustly recover the surface normal maps in the case of heavy specular reflection and outperforms the state-of-the-art techniques. (C) 2014 SPIE and IS&T
Improved Coding and Decoding Algorithm for Structured Light Illumination Scanning along Two-Direction

Sun Weilun,Xu Wen,Hu Dan,Liu Kai

DOI: https://doi.org/10.3788/lop222710

2023-01-01

Abstract:Multi -frequency -phase -shift structured light illumination scanning along two -direction will achieve higher robustness, but it increases time spending on scanning and computing. This paper proposes an improved coding and decoding strategy for scanning along two -direction by means of epipolar geometry. First, after scanning along vertical direction and getting unwrapped vertical phase, the temporary horizontal phase is obtained according to unwrapped vertical phase and epipolar geometry of system. Then, after scanning along vertical direction and getting wrapped horizontal phase, unwrapped horizontal phase is obtained by using temporary unwrap algorithm. Finally, a line -model is established between the camera and projector. When calculating a 3D point cloud, the intersection point of the camera line of sight and the projector line of sight is calculated, avoiding the traditional matrix inversion method and improving the point cloud calculation speed. The experimental results show that: 1) the root mean square error of final lateral phase is 4. 89 x 10-3; 2) the computation speeds of 3D point cloud after scanning along two -direction and scanning along one -direction are improved by 6. 08 times and 4. 10 times, respectively; 3) the proposed 3D point cloud reconstruction method for scanning along one -direction is within the error range of 10-11 mm compared with the traditional method.
Nonlinear Effect of the Structured Light Profilometry in the Phase-Shifting Method and Error Correction

Zhang Wan-Zhen,Chen Zhe-Bo,Xia Bin-Feng,Lin Bin,Cao Xiang-Qun

DOI: https://doi.org/10.1088/1674-1056/23/4/044212

2014-01-01

Abstract:Digital structured light（SL） profilometry is increasingly used in three-dimensional（3D） measurement technology.However, the nonlinearity of the off-the-shelf projectors and cameras seriously reduces the measurement accuracy. In this paper, first, we review the nonlinear effects of the projector–camera system in the phase-shifting structured light depth measurement method. We show that high order harmonic wave components lead to phase error in the phase-shifting method. Then a practical method based on frequency domain filtering is proposed for nonlinear error reduction. By using this method, the nonlinear calibration of the SL system is not required. Moreover, both the nonlinear effects of the projector and the camera can be effectively reduced. The simulations and experiments have verified our nonlinear correction method.
Epipolar Geometry Guided Highly Robust Structured Light 3D Imaging

Guijin Wang,Chenchen Feng,Xiaowei Hu,Hang Wang,Huazhong Yang

DOI: https://doi.org/10.1109/lsp.2021.3073266

2021-01-01

IEEE Signal Processing Letters

Abstract:Structured light (SL) based three-dimensional (3D) imaging technology has been widely employed in many fields of computer vision. However, currently available SL based depth sensors are sensitive to imaging noises, which severely limits the performance of subsequent advanced vision tasks. In this letter, we propose a robust and practical SL illumination pattern coding method based on epipolar geometry. The proposed pattern can effectively alleviate coding redundancy in traditional global random speckle SL patterns and make the stereo matching more robust to noise. Meanwhile, this coding strategy supplies sufficient non-local similar blocks, which inspires us to propose a uni-direction block-stacking 3D filtering algorithm to further improve the 3D imaging quality. To verify the proposed algorithms, we developed a prototype using the off-the-shelf projector and camera. Both simulation and real scene experimental results show that the proposed methods can achieve high-quality 3D imaging performance under different noisy conditions.
A Robust Surface Coding Method for Optically Challenging Objects Using Structured Light

Chi Zhang,Jing Xu,Ning Xi,Jianguo Zhao,Quan Shi

DOI: https://doi.org/10.1109/TASE.2013.2293576

2014-01-01

Abstract:Though the structured light measurement system has been successfully applied to the profile measurement of diffuse objects, it is still a challenge to measure shiny objects due to the mix of both specular and diffuse reflections. To this end, we propose a robust encoding and decoding method in this paper. First, the monochromatic stripe patterns are utilized to eliminate the effect of texture and color of objects. Second, an intensity mask, dynamically adjusting the intensity of a projected pattern, is applied to avoid overexposure without any pre-knowledge of the workpiece. Thus, it is more flexible and efficient, compared with the existing methods. Third, to solve the internal reflection of the shiny part, an extrapolation model, combined with the intensity mask, is developed to detect the stripe edge for pattern decoding, resulting in accurate and robust 3D reconstruction. Compared with traditional polarization based methods, it does not need to readjust for a new part. The experimental results show that the proposed method is capable of measuring various parts without surface pretreatment.
Structured Light Acquisition System Based on Waveband Separation

Liu Di,Yao Zhenning,Li Haifeng,Liu Xiangdong,Liu Xu

DOI: https://doi.org/10.3788/AOS201232.0612003

2012-01-01

Acta Optica Sinica

Abstract:Color-coded structural light system can reconstruct the 3D contour of the object in one shot by the using of three different patterns embedded into three color channels of the projector and camera.However,it cannot reconstruct a color object,and the crosstalks between different channels lower the accuracy.A novel color-coded structural light system is presented.By using multiple cameras and well designed beam splitter prism and filter,color crosstalk between different color channels in color-coded structural light system is eliminated.A process of two-shot acquiring color object prefile based on three-step phase measurement profilometry is analysed.Color calibration method is discussed to improve the system′s accuracy.After all the procedures,a colorful object can be reconstructed in two shots,which combines the speed and accuracy.
Improving Imaging Quality Using Structured Light Projection

Zhengjun Wu,Zhaofeng Cen,Xiaotong Li

DOI: https://doi.org/10.1117/12.2293997

2017-01-01

Abstract:Due to diffraction and aberrations, images will be degraded during the imaging process. It is needed to find a way to correct the aberration without changing the structure of imaging system. In this paper, a method for improving image quality and obtaining higher resolution is presented. The process of rebuilding high-resolution image from blurred image illuminated by structured light can be viewed as solving equations. The values of pixels in the clearer picture we want are the unknowns and the value of each pixel on the image plane is the weighted cumulant of them, of which the coefficient is decided by the point spared function of the imaging system. More non-equivalent equations are needed to get the solve because the coefficient-matrix is singular. In this passage, the equations mentioned above are the pictures with structured light. As shown by the simulation and experiment that the image quality is improved markedly. Compared with the image enhancement algorithms like Sobel sharpen, the algorithms with structured illumination can rebuild a more accurate image.
Line-encoded structured light measurement method in measuring shiny and transparent objects

Shichao Yang,Tian Yang,Yanxue Wu,Gaoxu Wu,Fei Liu

DOI: https://doi.org/10.1088/2040-8986/acbb8b

IF: 2.1

2023-02-15

Journal of Optics

Abstract:Structured Light Measurement (SLM) has been widely used in acquiring the three-dimensional (3D) shape of objects because of its high precision and high speed. However, the intensity based structured light coding strategies like sinusoidal patterns is difficult to measure transparent and shiny objects with high dynamic range (HDR) surface reflectance due to inaccurate light intensity information. Divergent multi-line structured light projection avoids this problem by extracting lines. But it requires a lot of gray-code patterns to unwrap the phase, which reduces the measurement efficiency. To overcome defects, this paper proposes a line-encoded few-pattern structured light measurement method in measuring shiny and transparent objects. In this method, we simultaneously extract the edge line, the center line and obtain their orders by projecting only five patterns. Many experiments were implemented to prove its accuracy and robustness. Because only five patterns are required, the proposed method greatly accelerate the process of reconstruction, which is indispensable for 3D real-time reconstruction. We conducted a dental model dynamic scanning experiment to prove it.

optics
A Novel Locally Adaptive Dynamic Programming Approach for Color Structured Light System.

Run Zou,Yu Zhou,Yao Yu,Sidan Du

DOI: https://doi.org/10.1007/978-3-642-33179-4_36

2012-01-01

Abstract:The authors present a color structured light system by projecting an optimized De Bruijn pattern for 3D model reconstruction under natural light condition. The main focus of this paper is to enhance the accuracy of the correspondence problem by designing a novel locally adaptive dynamic programming algorithm, which adjusts the support-weight of stripe in a given local window. The presented approach performs better in terms of smoothness and accuracy of construction result and is suitable for generating high-quality and real-time scans of moving object.
Vision Processing for Realtime 3-D Data Acquisition Based on Coded Structured Light

S. Y. Chen,Y. F. Li,Jianwei Zhang

DOI: https://doi.org/10.1109/tip.2007.914755

IF: 10.6

2008-02-01

IEEE Transactions on Image Processing

Abstract:Structured light vision systems have been successfully used for accurate measurement of 3-D surfaces in computer vision. However, their applications are mainly limited to scanning stationary objects so far since tens of images have to be captured for recovering one 3-D scene. This paper presents an idea for real-time acquisition of 3-D surface data by a specially coded vision system. To achieve 3-D measurement for a dynamic scene, the data acquisition must be performed with only a single image. A principle of uniquely color-encoded pattern projection is proposed to design a color matrix for improving the reconstruction efficiency. The matrix is produced by a special code sequence and a number of state transitions. A color projector is controlled by a computer to generate the desired color patterns in the scene. The unique indexing of the light codes is crucial here for color projection since it is essential that each light grid be uniquely identified by incorporating local neighborhoods so that 3-D reconstruction can be performed with only local analysis of a single image. A scheme is presented to describe such a vision processing method for fast 3-D data acquisition. Practical experimental performance is provided to analyze the efficiency of the proposed methods.

computer science, artificial intelligence,engineering, electrical & electronic
Optimal Structured Light a La Carte

Parsa Mirdehghan,Wenzheng Chen,Kiriakos N. Kutulakos

DOI: https://doi.org/10.1109/cvpr.2018.00654

2018-01-01

Abstract:We consider the problem of automatically generating sequences of structured-light patterns for active stereo triangulation of a static scene. Unlike existing approaches that use predetermined patterns and reconstruction algorithms tied to them, we generate patterns on the fly in response to generic specifications: number of patterns, projector-camera arrangement, workspace constraints, spatial frequency content, etc. Our pattern sequences are specifically optimized to minimize the expected rate of correspondence errors under those specifications for an unknown scene, and are coupled to a sequence-independent algorithm for perpixel disparity estimation. To achieve this, we derive an objective function that is easy to optimize and follows from first principles within a maximum-likelihood framework. By minimizing it, we demonstrate automatic discovery of pattern sequences, in under three minutes on a laptop, that can outperform state-of-the-art triangulation techniques.
A Robust and Accurate Structured Light Pattern for Dynamic 3D Reconstruction

Xiangyu Zhao,Songping Mai

DOI: https://doi.org/10.1117/12.2599681

2021-01-01

Abstract:A new method based on single shot using color coded image sequence for 3D reconstruction is proposed. The previous spatial neighborhood coding methods often need to determine the center of the pattern, but inaccurate positioning of pattern feature points will bring errors to 3D reconstruction. In this paper, a color-coded pattern is designed as two opposite triangular patterns to directly establish the center point of the pattern, thus avoiding the error caused by the inaccurate positioning of the center point after the pattern distortion. We designed a corner detector for our feature point detection, it is efficient and accurate. The pattern is coded in four colors of red, blue, green and pale yellow. We use a quaternary sixth order primitive polynomial to generate the pseudorandom sequence, which is then regenerated into the M-array. The final size of the code pattern is 65×63, and the unique code window size is 2×3. Experiments show that the 3D reconstruction owns high measurement precision and the decoding has strong robustness.
Structured Light: Peculiar Properties and Applications

O. Angelsky,C. Zenkova,Wenjun Yan

DOI: https://doi.org/10.1117/12.2552916

2020-01-01

Abstract:This paper is devoted to the analysis of the state of physical (optical) science, highlighting the prospects for the further development of the most relevant trends that are embodied in communication systems, micro- and nanobiology, nanomedicine, high-tech communication devices, macro- and microphysics.
High-resolution 3D reconstruction for complex color scenes with structured light

WeiDong Hu,Mingying Gong,Yanhui Hong,Lifeng Sun,Shiqiang Yang

DOI: https://doi.org/10.1109/ICMEW.2014.6890610

2014-01-01

Abstract:High-resolution 3D reconstruction has become increasingly important due to its wide applications such as 3D printing, entertainment, engineering etc. In this paper, a high-resolution 3D reconstruction system based on structured light is proposed. Our proposed system contains a consuming projector and a digital camera. Unlike the traditional structured light which directly projects patterns into scenes, we project extended patterns generated from a 2D mother pattern and analysis the luminance changes of pixels of captured images to restore the mother pattern. With the help of extended patterns, the proposed system can not only deal with complex scenes robustly, but also deal with color scenes. And since the positions are matched based on 2D perfect map, external parameters of the projector and digital camera are not required during the evaluation. Experimental results show that the proposed system can generates high-resolution surfaces.
Towards higher-dimensional structured light

Chao He,Yijie Shen,Andrew Forbes

DOI: https://doi.org/10.1038/s41377-022-00897-3

2022-07-06

Light: Science and Applications

Abstract:Structured light refers to the arbitrarily tailoring of optical fields in all their degrees of freedom (DoFs), from spatial to temporal. Although orbital angular momentum (OAM) is perhaps the most topical example, and celebrating 30 years since its connection to the spatial structure of light, control over other DoFs is slowly gaining traction, promising access to higher-dimensional forms of structured light. Nevertheless, harnessing these new DoFs in quantum and classical states remains challenging, with the toolkit still in its infancy. In this perspective, we discuss methods, challenges, and opportunities for the creation, detection, and control of multiple DoFs for higher-dimensional structured light. We present a roadmap for future development trends, from fundamental research to applications, concentrating on the potential for larger-capacity, higher-security information processing and communication, and beyond.
Dual-projector structured light 3D shape measurement.

Kai Liu,D. Lau,Ying Yu,Matthew P. Ruffner

DOI: https://doi.org/10.1364/ao.378363

IF: 1.9

2020-02-01

Applied Optics

Abstract:Structured light illumination is an active three-dimensional scanning technique that uses a projector and camera pair to project and capture a series of stripe patterns; however, with a single camera and single projector, structured light scanning has issues associated with scan occlusions, multi-path, and weak signal reflections. To address these issues, this paper proposes dual-projector scanning using a range of projector/camera arrangements. Unlike previous attempts at dual-projector scanning, the proposed scanner drives both light engines simultaneously, using temporal-frequency multiplexing to computationally decouple the projected patterns. Besides presenting the details of how such a system is built, we also present experimental results demonstrating how multiple projectors can be used to (1) minimize occlusions; (2) achieve higher signal-to-noise ratios having twice a single projector's brightness; (3) reduce the number of component video frames required for a scan; and (4) detect multi-path interference.

Computer Science,Medicine,Engineering
High Dynamic Range Structured Light 3-D Measurement Based on Region Adaptive Fringe Brightness

Yichen Fu,Junfeng Fan,Fengshui Jing,Min Tan

DOI: https://doi.org/10.1109/tie.2023.3303655

IF: 7.7

2023-01-01

IEEE Transactions on Industrial Electronics

Abstract:Fringe projection profilometry is one of the main methods for active 3-D measurement, but practical problems, such as how to obtain a complete point cloud by overcoming the effects of high surface reflectivity of objects, remain to be solved. So, in this article, we propose a set of hardware devices and region adaptive structured light algorithms. First, we introduce the surface structured light measurement system adopted in this article. Second, based on the accurate optical modeling of the measurement scene, the principles for generating the adaptive optimal projection brightness are derived in detail and the process is accelerated by exploiting the speed of neural network. Meanwhile, we creatively use the chain code combined with the M-estimator sample consensus method to find the homography from the saturated region of the camera plane to the corresponding region of the projector plane for generating fringe images with adaptive brightness. Furthermore, we chose the more robust line-shifting code and binarization method for subpixel 3-D reconstruction. Experiments have shown that our work is efficient, superior, and accurate in measuring objects with high reflective surface properties.

automation & control systems,engineering, electrical & electronic,instruments & instrumentation
A Three-Dimensional Structured Light Vision System by Using a Combination of Single-Line and Three-Line Lasers

Qiucheng Sun,Zeming Ren,Jinlong Zhu,Weiyu Dai,Mingze Wang,Mingyu Sun

DOI: https://doi.org/10.3390/s23010013

IF: 3.9

2022-12-21

Sensors

Abstract:A multi-line structured light measurement method that combines a single-line and a three-line laser, in which precision sliding rails and displacement measurement equipment are not required, is proposed in this paper. During the measurement, the single-line structured light projects onto the surface of an object and the three-line structured light remains fixed. The single-line laser is moved and intersects with the three-line laser to form three intersection points. The single-line light plane can be solved using the camera coordinates of three intersection points, thus completing the real-time calibration of the scanned light plane. The single-line laser can be scanned at any angle to determine the overall complete three-dimensional (3D) shape of the object during the process. Experimental results show that this method overcomes the difficulty of obtaining information about certain angles and locations and can effectively recover the 3D shape of the object. The measurement system's repetition error is under 0.16 mm, which is sufficient to measure the 3D shapes of complicated workpieces.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

Robust structured light with efficient redundant codes

Structured Light with Redundancy Codes

Target Enhanced 3D Reconstruction Based on Polarization-Coded Structured Light.

Robust photometric stereo using structural light sources

Improved Coding and Decoding Algorithm for Structured Light Illumination Scanning along Two-Direction

Nonlinear Effect of the Structured Light Profilometry in the Phase-Shifting Method and Error Correction

Epipolar Geometry Guided Highly Robust Structured Light 3D Imaging

A Robust Surface Coding Method for Optically Challenging Objects Using Structured Light

Structured Light Acquisition System Based on Waveband Separation

Improving Imaging Quality Using Structured Light Projection

Line-encoded structured light measurement method in measuring shiny and transparent objects

A Novel Locally Adaptive Dynamic Programming Approach for Color Structured Light System.

Vision Processing for Realtime 3-D Data Acquisition Based on Coded Structured Light

Optimal Structured Light a La Carte

A Robust and Accurate Structured Light Pattern for Dynamic 3D Reconstruction

Structured Light: Peculiar Properties and Applications

High-resolution 3D reconstruction for complex color scenes with structured light

Towards higher-dimensional structured light

Dual-projector structured light 3D shape measurement.

High Dynamic Range Structured Light 3-D Measurement Based on Region Adaptive Fringe Brightness

A Three-Dimensional Structured Light Vision System by Using a Combination of Single-Line and Three-Line Lasers