1D speckle-learned structured light recognition

Purnesh Singh Badavath,Venugopal Raskatla,Vijay Kumar

DOI: https://doi.org/10.1364/ol.514739

IF: 3.6

2024-02-15

Optics Letters

Abstract:Purnesh Singh Badavath, Venugopal Raskatla, Vijay Kumar In this Letter, we introduce a novel, to the best of our knowledge, structured light recognition technique based on the 1D speckle ... [Opt. Lett. 49, 1045-1048 (2024)]

optics

What problem does this paper attempt to address?

Machine learning meets Singular Optics: Speckle-based Structured light demultiplexing

Venugopal Raskatla,Purnesh Singh Badavath,Vijay Kumar

2023-11-01

Abstract:In this paper, the advancements in structured light beams recognition using speckle-based convolutional neural networks (CNNs) have been presented. Speckle fields, generated by the interference of multiple wavefronts diffracted and scattered through a diffuser, project a random distribution. The generated random distribution of phase and intensity correlates to the structured light beam of the corresponding speckle field. This unique distribution of phase and intensity offers an additional dimension for recognizing the encoded information in structured light. The CNNs are well-suited for harnessing this unique ability to recognize the speckle field by learning hidden patterns within data. One notable advantage of speckle-based recognition is their ability to identify structured light beams from a small portion of the speckle field, even in high-noise environments. The diffractive nature of the speckle field enables off-axis recognition, showcasing its capability in information broadcasting employing structured light beams. This is a significant departure from direct-mode detection-based models to alignment-free speckle-based detection models, which are no longer constrained by the directionality of laser beams.

Optics
Imaging and tracking through scattering medium with low bit depth speckle

Qinghuan Xu,Baoqing Sun,Jia Zhao,Zuojia Wang,Liuge Du,Chonglei Sun,Xianye Li,Xun Li

DOI: https://doi.org/10.1109/JPHOT.2020.2974898

IF: 2.4

2020-01-01

IEEE Photonics Journal

Abstract:Optical imaging/tracking through scattering medium is a difficult challenge. We demonstrate a method using speckles with multiple bit depths to image objects through scattering medium. The image of the object can be reconstructed even if the speckles are binary. Speckles with different bit depths are obtained by quantifying the original speckle. And we discuss the application of binarized speckles...
Structured Light: Peculiar Properties and Applications

O. Angelsky,C. Zenkova,Wenjun Yan

DOI: https://doi.org/10.1117/12.2552916

2020-01-01

Abstract:This paper is devoted to the analysis of the state of physical (optical) science, highlighting the prospects for the further development of the most relevant trends that are embodied in communication systems, micro- and nanobiology, nanomedicine, high-tech communication devices, macro- and microphysics.
Seeing the Invisible through Speckle Images

Weiru Fan,Xiaobin Tang,Xingqi Xu,Huizhu Hu,Vladislav V. Yakovlev,Shi-Yao Zhu,Da-Wei Wang,Delong Zhang

2024-09-27

Abstract:Scattering obscures information carried by wave by producing a speckle pattern, posing a common challenge across various fields, including microscopy and astronomy. Traditional methods for extracting information from speckles often rely on significant physical assumptions, complex devices, or intricate algorithms. Recently, machine learning has emerged as a scalable and widely adopted tool for interpreting speckle patterns. However, most current machine learning techniques depend heavily on supervised training with extensive labeled datasets, which is problematic when labels are unavailable. To address this, we propose a strategy based on unsupervised learning for speckle recognition and evaluation, enabling to capture high-level information, such as object classes, directly from speckles without labeled data. By deriving invariant features from speckles, this method allows for the classification of speckles and facilitates diverse applications in image sensing. We experimentally validated our strategy through two significant applications: a noninvasive glucose monitoring system capable of differentiating time-lapse glucose concentrations, and a high-throughput communication system utilizing multimode fibers in dynamic environments. The versatility of this method holds promise for a broad range of far-reaching applications, including biomedical diagnostics, quantum network decoupling, and remote sensing.

Optics
One-Shot Scanning Using a Color Stripe Pattern

Renju Li,Hongbin Zha

DOI: https://doi.org/10.1109/ICPR.2010.412

IF: 8

2010-01-01

Pattern Recognition

Abstract:Structured light 3D scanning has many applications such as 3D modeling, animation, motion analysis, deformation measurement and so on. Traditional structured light methods make use of a sequence of patterns to obtain the dense 3D data of objects. However, few methods have been proposed to achieve pixel wise reconstruction using one pattern only. In this paper, we proposes a one-shot scanning system based on a novel stripe pattern. This pattern uses color stripes with quadratic intensity distribution in each stripe. The color distribution is based on a De Bruijn sequence with six colors and order three. Graph cut is utilized to decode the color information and the resulting code is calculated using local intensity. Compared with traditional methods, the proposed method uses one pattern only and achieves pixel wise reconstruction. Experimental results show that our one-shot scanning system can robustly capture 3D data with high accuracy.
Deep-learned speckle pattern and its application to ghost imaging

Xiaoyu Nie,Haotian Song,Wenhan Ren,Xingchen Zhao,Zhedong Zhang,Tao Peng,Marlan O. Scully

DOI: https://doi.org/10.48550/arXiv.2112.13293

2021-12-28

Abstract:In this paper, we present a method for speckle pattern design using deep learning. The speckle patterns possess unique features after experiencing convolutions in Speckle-Net, our well-designed framework for speckle pattern generation. We then apply our method to the computational ghost imaging system. The standard deep learning-assisted ghost imaging methods use the network to recognize the reconstructed objects or imaging algorithms. In contrast, this innovative application optimizes the illuminating speckle patterns via Speckle-Net with specific sampling ratios. Our method, therefore, outperforms the other techniques for ghost imaging, particularly its ability to retrieve high-quality images with extremely low sampling ratios. It opens a new route towards nontrivial speckle generation by referring to a standard loss function on specified objectives with the modified deep neural network. It also has great potential for applications in the fields of dynamic speckle illumination microscopy, structured illumination microscopy, x-ray imaging, photo-acoustic imaging, and optical lattices.

Image and Video Processing,Optics
DSCNet: lightweight and efficient self-supervised network via depthwise separable cross convolution blocks for speckle image matching

Lin Li,Peng Wang,Lingrui Wang,Changku Sun,Fu Luhua,Luhua Fu

DOI: https://doi.org/10.1364/oe.519957

IF: 3.8

2024-03-10

Optics Express

Abstract:Lin Li, Peng Wang, Lingrui Wang, Changku Sun, Luhua Fu Speckle structured light has become a research hotspot due to its ability to acquire target three-dimensional information with single image ... [Opt. Express 32, 10715-10731 (2024)]

optics
Speckle Structured-Light-Based Three-Dimensional Imaging Technology and Its Sensor Design Using VCSEL Projection Array

Wei Yin,Mingyu Li,Yan Hu,Shijie Feng,Xiaolei Zhang,Huai Wang,Qian Chen,Chao Zuo

DOI: https://doi.org/10.3788/lop230428

2023-01-01

Laser & Optoelectronics Progress

Abstract:Speckle projection profilometry (SPP) encodes depth information of a scene by projecting a single random speckle pattern and establishes the global correspondences between stereo images using speckle matching technology, thereby achieving a single-shot three-dimensional (3D) reconstruction. However, it is still difficult to encode a globally unique feature for every pixel in an entire measurement space by projecting only a speckle pattern due to the measured object surface with complex reflection characteristics and the perspective differences between stereo cameras. The resulting mismatching problem leads to the low measurement accuracy of SPP, making it difficult to meet the highprecision measurement requirements of some industrial scenarios. In this study, we propose a speckle structured-lightbased 3D imaging technique and its sensor design method using a vertical-cavity surface-emitting laser ( VCSEL) projection array. The proposed 3D structured- light sensor integrates three miniaturized speckle projection modules to project a set of speckle patterns with different spatial positions, spatially and temporally encoding the depth information of the measured scene efficiently. In addition, a coarse-to- fine spatial-temporal speckle correlation algorithm is proposed to improve the measurement accuracy and reconstruct the fine shapes of complex objects. Experiments, including precision analysis, 3D model scanning, detection of small target metal parts, and measurement of complex scenes, indicate that the proposed 3D structured-light sensor can realize high-precision 3D measurements at a long distance over a large field of view, which can further be applied to industrial scenarios such as part sorting and robot stacking.
Deep-learning-based recognition of multi-singularity structured light

Hao Wang,Xilin Yang,Zeqi Liu,Jing Pan,Yuan Meng,Zijian Shi,Zhensong Wan,Hengkang Zhang,Yijie Shen,Xing Fu,Qiang Liu

DOI: https://doi.org/10.1515/nanoph-2021-0489

IF: 7.5

2021-10-14

Nanophotonics

Abstract:Abstract Structured light with customized topological patterns inspires diverse classical and quantum investigations underpinned by accurate detection techniques. However, the current detection schemes are limited to vortex beams with a simple phase singularity. The precise recognition of general structured light with multiple singularities remains elusive. Here, we report deep learning (DL) framework that can unveil multi-singularity phase structures in an end-to-end manner, after feeding only two intensity patterns upon beam propagation. By outputting the phase directly, rich and intuitive information of twisted photons is unleashed. The DL toolbox can also acquire phases of Laguerre–Gaussian (LG) modes with a single singularity and other general phase objects likewise. Enabled by this DL platform, a phase-based optical secret sharing (OSS) protocol is proposed, which is based on a more general class of multi-singularity modes than conventional LG beams. The OSS protocol features strong security, wealthy state space, and convenient intensity-based measurements. This study opens new avenues for large-capacity communications, laser mode analysis, microscopy, Bose–Einstein condensates characterization, etc.

optics,physics, applied,materials science, multidisciplinary,nanoscience & nanotechnology
Structured-light 3D Shape Measurements Using Deep Learning

Shijie Feng,Chao Zuo,Qian Chen

DOI: https://doi.org/10.1117/12.2592706

2021-01-01

Abstract:Over the past few decades, tremendous efforts have been devoted to developing various techniques for fringe analysis, and they can be broadly classified into two categories: (1) phase-shifting (PS) methods which require multiple fringe patterns to extract phase information and (2) spatial phase demodulation methods which allow phase retrieval from a single fringe pattern, such as the Fourier transform (FT), windowed Fourier transform (WFT), and wavelet transform (WT) methods. Compared with spatial phase demodulation methods, the multiple-shot phase-shifting techniques are generally more robust and can achieve pixel-wise phase measurement with higher resolution and accuracy. Furthermore, the phase-shifting measurements are quite insensitive to non-uniform background intensity and fringe modulation. Nevertheless, due to their multi-shot nature, these methods are difficult to apply to dynamic measurements and are more susceptible to external disturbance and vibrations. Thus, for many applications, phase extraction from a single fringe pattern is desired, which falls under the purview of spatial fringe analysis. Recently, we demonstrated that the use of convolutional neural networks can substantially enhance the accuracy of phase demodulation from a single fringe pattern. Moreover, we find the powerful learning ability of deep neural network (DNN) enables the phase unwrapping, super-fast 3D shape measurement of transient events, multi-view fringe projection and so on. From comparative results, the DNN shows better performance over traditional state-of-the-art methods in terms of the phase accuracy and efficiency. We believe the deep learning technique is a powerful technique to handle fringe images and will find wide applications in 3D measurements with structured-light illumination.
Addressable Structured Light System Using Metasurface Optics and an Individually Addressable VCSEL Array

Chenyang Wu,Xuanlun Huang,Yipeng Ji,Tingyu Cheng,Jiaxing Wang,Nan Chi,Shaohua Yu,Connie J. Chang-Hasnain

DOI: https://doi.org/10.1364/prj.516942

IF: 7.6

2024-01-01

Photonics Research

Abstract:Structured -light (SL) based 3D sensors have been widely used in many fields. Speckle SL is the most widely deployed among all SL sensors due to its light weight, compact size, fast video rate, and low cost. The transmitter (known as the dot projector) consists of a randomly patterned vertical -cavity surface -emitting laser (VCSEL) array multiplicated by a diffractive optical element (DOE) with a fixed repeated pattern. Given that the separation of any two speckles is only one known and fixed number (albeit random), there are no other known scales to calibrate or average. Hence, typical SL sensors require extensive in -factory calibrations, and the depth resolution is limited to 1 mm at similar to 60 cm distance. In this paper, to the best of our knowledge, we propose a novel dot projector and a new addressable SL (ASL) 3D sensor by using a regularly spaced, individually addressable VCSEL array, multiplicated by a metasurface-DOE (MDOE) into a random pattern of the array. Dynamically turning on or off the VCSELs in the array provides multiple known distances between neighboring speckles, which is used as a " built-in caliper " to achieve higher accuracy of depth. Serving as a precise " vernier caliper, " the addressable VCSEL array enables fine control over speckle positions and high detection precision. We experimentally demonstrated that the proposed method can result in sub -hundred -micron level precision. This new concept opens new possibilities for applications such as 3D computation, facial recognition, and wearable devices. (c) 2024 Chinese Laser Press
A 3D Reconstruction Method Based on Homogeneous De Bruijn-Encoded Structured Light

Weimin Li,Songlin Li

DOI: https://doi.org/10.3390/photonics11050458

IF: 2.536

2024-05-15

Photonics

Abstract:Structured light three-dimensional reconstruction is one of the important methods for non-contact acquisition of sparse texture object surfaces. Variations in ambient illumination and disparities in object surface reflectance can significantly impact the fidelity of three-dimensional reconstruction, introducing considerable inaccuracies. We introduce a robust method for color speckle structured light encoding, which is based on a variant of the De Bruijn sequence, termed the Homogeneous De Bruijn Sequence. This innovative approach enhances the reliability and accuracy of structured light techniques for three-dimensional reconstruction by utilizing the distinctive characteristics of Homogeneous De Bruijn Sequences. Through a pruning process applied to the De Bruijn sequence, a structured light pattern with seven distinct color patches is generated. This approach ensures a more equitable distribution of speckle information.

optics
Machine learning for classifying and interpreting coherent X-ray speckle patterns

Mingren Shen,Dina Sheyfer,Troy David Loeffler,Subramanian K.R.S. Sankaranarayanan,G. Brian Stephenson,Maria K. Y. Chan,Dane Morgan

2023-09-02

Abstract:Speckle patterns produced by coherent X-ray have a close relationship with the internal structure of materials but quantitative inversion of the relationship to determine structure from speckle patterns is challenging. Here, we investigate the link between coherent X-ray speckle patterns and sample structures using a model 2D disk system and explore the ability of machine learning to learn aspects of the relationship. Specifically, we train a deep neural network to classify the coherent X-ray speckle patterns according to the disk number density in the corresponding structure. It is demonstrated that the classification system is accurate for both non-disperse and disperse size distributions.

Materials Science,Computer Vision and Pattern Recognition,Machine Learning
Direct Object Recognition Without Line-of-Sight Using Optical Coherence

Xin Lei,Liangyu He,Yixuan Tan,Ken Xingze Wang,Xinggang Wang,Yihan Du,Shanhui Fan,Zongfu Yu

DOI: https://doi.org/10.48550/arXiv.1903.07705

2019-03-19

Abstract:Visual object recognition under situations in which the direct line-of-sight is blocked, such as when it is occluded around the corner, is of practical importance in a wide range of applications. With coherent illumination, the light scattered from diffusive walls forms speckle patterns that contain information of the hidden object. It is possible to realize non-line-of-sight (NLOS) recognition with these speckle patterns. We introduce a novel approach based on speckle pattern recognition with deep neural network, which is simpler and more robust than other NLOS recognition methods. Simulations and experiments are performed to verify the feasibility and performance of this approach.

Computer Vision and Pattern Recognition,Optics
Optical Machine Learning with Incoherent Light and a Single-Pixel Detector

Shuming Jiao,Jun Feng,Yang Gao,Ting Lei,Zhenwei Xie,Xiaocong Yuan

DOI: https://doi.org/10.1364/ol.44.005186

IF: 3.6

2019-01-01

Optics Letters

Abstract:An optical diffractive neural network (DNN) can be implemented with a cascaded phase mask architecture. Like an optical computer, the system can perform machine learning tasks such as number digit recognition in an all-optical manner. However, the system can work only under coherent light illumination, and the precision requirement in practical experiments is quite high. This Letter proposes an optical machine learning framework based on single-pixel imaging (MLSPI). The MLSPI system can perform the same linear pattern recognition task as DNN. Furthermore, it can work under incoherent lighting conditions, has lower experimental complexity, and can be easily programmable.
Harnessing disordered photonics via multi-task learning towards intelligent four-dimensional light field sensors

Sheng-ke Zhu,Ze-huan Zheng,Weijia Meng,Shan-shan Chang,Yingling Tan,Lu-Jian Chen,Xinyuan Fang,Min Gu,Jin-hui Chen,Min Gu and Jin-hui Chen

DOI: https://doi.org/10.1186/s43074-023-00102-7

IF: 16.5

2023-08-17

PhotoniX

Abstract:The complete description of a continuous-wave light field includes its four fundamental properties: wavelength, polarization, phase and amplitude. However, the simultaneous measurement of a multi-dimensional light field of such four degrees of freedom is challenging in conventional optical systems requiring a cascade of dispersive and polarization elements. In this work, we demonstrate a disordered-photonics-assisted intelligent four-dimensional light field sensor. This is achieved by discovering that the speckle patterns, generated from light scattering in a disordered medium, are intrinsically sensitive to a high-dimension light field given their high structural degrees of freedom. Further, the multi-task-learning deep neural network is leveraged to process the single-shot light-field-encoded speckle images free from any prior knowledge of the complex disordered structures and realizes the high-accuracy recognition of full-Stokes vector, multiple orbital angular momentum (OAM), wavelength and power. The proof-of-concept study shows that the states space of four-dimensional light field spanning as high as 1680=4 (multiple-OAM) 2 (OAM power spectra) 15 (multiple-wavelength) 14 (polarizations) can be well recognized with high accuracy in the chip-integrated sensor. Our work provides a novel paradigm for the design of optical sensors for high-dimension light fields, which can be widely applied in optical communication, holography, and imaging.

optics
Towards higher-dimensional structured light

Chao He,Yijie Shen,Andrew Forbes

DOI: https://doi.org/10.1038/s41377-022-00897-3

2022-07-06

Light: Science and Applications

Abstract:Structured light refers to the arbitrarily tailoring of optical fields in all their degrees of freedom (DoFs), from spatial to temporal. Although orbital angular momentum (OAM) is perhaps the most topical example, and celebrating 30 years since its connection to the spatial structure of light, control over other DoFs is slowly gaining traction, promising access to higher-dimensional forms of structured light. Nevertheless, harnessing these new DoFs in quantum and classical states remains challenging, with the toolkit still in its infancy. In this perspective, we discuss methods, challenges, and opportunities for the creation, detection, and control of multiple DoFs for higher-dimensional structured light. We present a roadmap for future development trends, from fundamental research to applications, concentrating on the potential for larger-capacity, higher-security information processing and communication, and beyond.
DeepSCI: scalable speckle correlation imaging using physics-enhanced deep learning

Zhiwei Tang,Fei Wang,Shanshan Zheng,Ying Jin,Guohai Situ,Zhenfeng Fu

DOI: https://doi.org/10.1364/ol.484867

IF: 3.6

2023-04-21

Optics Letters

Abstract:Zhiwei Tang, Fei Wang, ZhenFeng Fu, Shanshan Zheng, Ying Jin, Guohai Situ In this Letter we present a physics-enhanced deep learning approach for speckle correlation imaging (SCI), i.e., DeepSCI. DeepSCI ... [Opt. Lett. 48, 2285-2288 (2023)]

optics
Deep-learning-assisted optical communication with discretized state space of structured light

Minyang Zhang,Dong-Xu Chen,Pengxiang Ruan,Jun Liu,Jun-Long Zhao,Chui-Ping Yang

2024-04-19

Abstract:The rich structure of transverse spatial modes of structured light has facilitated their extensive applications in quantum information and optical communication. The Laguerre-Gaussian (LG) modes, which carry a well-defined orbital angular momentum (OAM), consist of a complete orthogonal basis describing the transverse spatial modes of light. The application of OAM in free-space optical communication is restricted due to the experimentally limited OAM numbers and the complex OAM recognition methods. Here, we present a novel method that uses the advanced deep learning technique for LG modes recognition. By discretizing the spatial modes of structured light, we turn the OAM state regression into classification. A proof-of-principle experiment is also performed, showing that our method effectively categorizes OAM states with small training samples and high accuracy. By assigning each category a classical information, we further apply our approach to an image transmission task, demonstrating the ability to encode large data with low OAM number. This work opens up a new avenue for achieving high-capacity optical communication with low OAM number based on structured light.

Optics,Quantum Physics
Extraction and Segmentation Method of Laser Stripe in Linear Structured Light Scanner

Zhuo-Ren Wan,Lei-Jie Lai,Jian Mao,Li-Min Zhu

DOI: https://doi.org/10.1117/1.oe.60.4.046104

IF: 1.3

2021-01-01

Optical Engineering

Abstract:Line structured light scanning technology is extensively applied in three-dimensional (3D) reconstruction due to its advantages of having high precision and being non-contact. To quickly and accurately obtain the feature information of the light stripe center, a method of extraction and adaptive segmentation of the light stripe center is proposed for 3D measurement of structured light. First, a subpixel extraction method is presented to extract the light strip center; it quickly determines the normal direction of the light stripe and accurately extracts the light stripe center in the normal direction by employing the improved gray gravity method. Second, an adaptive contour segmentation method based on slope difference and dynamic neighborhood optimization is presented. Finally, the proposed method is adopted to measure the geometric size of the complex manufactured parts. The experimental results demonstrate that the maximum measurement error is 0.15 mm and the accuracy is <1%, indicating that the method is accurate enough to be applied to various kinds of measurements. (C) 2021 Society of Photo-Optical Instrumentation Engineers (SPIE)

1D speckle-learned structured light recognition

Machine learning meets Singular Optics: Speckle-based Structured light demultiplexing

Imaging and tracking through scattering medium with low bit depth speckle

Structured Light: Peculiar Properties and Applications

Seeing the Invisible through Speckle Images

One-Shot Scanning Using a Color Stripe Pattern

Deep-learned speckle pattern and its application to ghost imaging

DSCNet: lightweight and efficient self-supervised network via depthwise separable cross convolution blocks for speckle image matching

Speckle Structured-Light-Based Three-Dimensional Imaging Technology and Its Sensor Design Using VCSEL Projection Array

Deep-learning-based recognition of multi-singularity structured light

Structured-light 3D Shape Measurements Using Deep Learning

Addressable Structured Light System Using Metasurface Optics and an Individually Addressable VCSEL Array

A 3D Reconstruction Method Based on Homogeneous De Bruijn-Encoded Structured Light

Machine learning for classifying and interpreting coherent X-ray speckle patterns

Direct Object Recognition Without Line-of-Sight Using Optical Coherence

Optical Machine Learning with Incoherent Light and a Single-Pixel Detector

Harnessing disordered photonics via multi-task learning towards intelligent four-dimensional light field sensors

Towards higher-dimensional structured light

DeepSCI: scalable speckle correlation imaging using physics-enhanced deep learning

Deep-learning-assisted optical communication with discretized state space of structured light

Extraction and Segmentation Method of Laser Stripe in Linear Structured Light Scanner