Machine learning enhanced smart tactile specklegram fiber sensor using spatial mode downsampling

Xiao Liu,Sijin He,Juan Kang,Bo Liu,Chen Zhu

DOI: https://doi.org/10.1364/oe.532891

IF: 3.8

2024-08-18

Optics Express

Abstract:Xiao Liu, Sijin He, Juan Kang, Bo Liu, Chen Zhu This study presents a proof-of-concept demonstration of a demodulation technique using a seven-core fiber (SCF) and machine learning (ML) ... [Opt. Express 32, 31783-31796 (2024)]

optics

What problem does this paper attempt to address?

Ameliorated Spatial Division Multiplexing System of FBG Sensors Based on Novel Wavelength Detection and Demodulation Technique

C Rong,HP Wang,LN Liu,B Lin,SS Xie,M Cheng

DOI: https://doi.org/10.1117/12.571250

2005-01-01

Abstract:A novel wavelength detection and demodulation technique of fiber Bragg gratings sensors is put forward. The principle, experimental setup and results of the sensor are analyzed. The new technique converts detections of shift of center reflecting wavelength into detection of light intensity extremum. Furthermore, a spatial division multiplexing system of Bragg gratings sensors based on the novel technique is developed. The multiplexing numbers are not confined by bandwidth of light source and tunable scope of single fiber Bragg grating.
Multipoint and Multi-physical Quantity Sensing by Multimode Fiber with Demodulation of Deep Learning

Miaofeng Fu,Jun Zheng,Shenyi Liu,Yunxu Sun,Wei Liu

DOI: https://doi.org/10.1109/icocn59242.2023.10236362

2023-01-01

Abstract:A multimode fiber (MMF) speckle sensor based on deep learning is demonstrated for sensing multi-physical quantity at different position. The mode fields at the end of MMF compose speckle patterns, which are sensitive to the various types of disturbing with different manners. Further, the MMF also carry numerous information, which make it possible sensing different physical quantity at different points in just one MMF. Here, it is demonstrated to sense stretching, bending, and temperature in a MMF of 0.5m by the deep learning methods. Its stretching precision is up to 0.005mm with the U-Net.
Application of machine learning in optical fiber sensors

Yifan Zhou,Ya-nan Zhang,Qi Yu,Lirong Ren,Qi Liu,Yong Zhao

DOI: https://doi.org/10.1016/j.measurement.2024.114391

IF: 5.6

2024-03-01

Measurement

Abstract:In recent years, with the increasing demand for intelligent society, intelligent photonics has developed rapidly. Machine learning (ML), as a subset of artificial intelligence (AI), has played an important role in the intelligent evolution of optical fiber sensors. Its impact extends beyond enhancing sensor performance by introducing innovative problem-solving approaches. Specifically, ML algorithms have become instrumental in signal demodulation and elevating the efficacy of discrete and distributed sensors, and have also greatly promoted the development of optical fiber speckle pattern processing. This paper presents the latest advancements in ML-based optical fiber sensors, outlines the problems faced by conventional demodulation methods and the common ML algorithms applied in optical fiber sensors, and emphasizes key applications. Additionally, this paper delves into the challenges and future development of this emerging research direction.

engineering, multidisciplinary,instruments & instrumentation
Spatially Variant Deblur and Image Enhancement in a Single Multimode Fiber Imaged by Deep Learning

Qilin Deng,Zhong Wen,Zhenyu Dong,Jianbin Tang,Wei Chen,Xu Liu,Qing Yang

DOI: https://doi.org/10.1364/ol.469034

IF: 3.6

2022-01-01

Optics Letters

Abstract:A single multimode fiber has been applied in minimally invasive endoscopy with wavefront shaping for biological research such as brain imaging. Most of the fibers, such as step-index and graded-index multimode fibers, give rise to spatially variant blur due to limits on the numerical aperture and collection efficiency. Routines to solve this problem are based on iterative algorithms, which are often slow and computer-intense. We developed a method to synthesize datasets for driving a deep learning network to deblur and denoise the spatially variant degraded image. This approach is fast (5 ms), up to three orders of magnitude faster than the iterative way. Furthermore, our method can be applied to different types of fiber endoscopy, and two types of fiber are tested here. The performance is verified on fluorescence beads and three kinds of biological tissue sections in the experiment, demonstrating effectiveness in image enhancement.
Multimode optical fiber sensors: from conventional to machine learning-assisted

Kun Wang,Yosuke Mizuno,Xingchen Dong,Wolfgang Kurz,Michael Köhler,Patrick Kienle,Heeyoung Lee,Martin Jakobi,Alexander W Koch,Michael Koehler,A W Koch

DOI: https://doi.org/10.1088/1361-6501/ad0318

IF: 2.398

2023-10-15

Measurement Science and Technology

Abstract:Multimode fiber (MMF) sensors have been extensively developed and utilized in various sensing applications for decades. Traditionally, the performance of MMF sensors was improved by conventional methods that focused on structural design and specialty fibers. However, in recent years, the blossom of machine learning techniques has opened up new avenues for enhancing the performance of MMF sensors. Unlike conventional methods, machine learning techniques do not require complex structures or rare specialty fibers, which reduces fabrication difficulties and lowers costs. In this review, we provide an overview of the latest developments in MMF sensors, ranging from conventional methods to those assisted by machine learning. This article begins by categorizing MMF sensors based on their sensing applications, including temperature and strain sensors, displacement sensors, refractive index sensors, curvature sensors, bio/chemical sensors, and other sensors. Their distinct sensor structures and sensing properties are thoroughly reviewed. Subsequently, the machine learning-assisted MMF sensors that have been recently reported are analyzed and categorized into two groups: learning the specklegrams and learning the spectra. The review provides a comprehensive discussion and outlook on MMF sensors, concluding that they are expected to be utilized in a wide range of applications.

engineering, multidisciplinary,instruments & instrumentation
A rapid demodulation method for the interference spectrum of optical fiber sensors based on machine learning

Xiao Yue-Yu,Xu Xin-Yu,Lai Yan-Xiang

DOI: https://doi.org/10.1088/1555-6611/ad7c39

IF: 1.2

2024-09-27

Laser Physics

Abstract:Interferometric optical fiber sensors based on wavelength demodulation play an important role in the field of high-precision measurements. Generally used wavelength tracking method is based on the locations of feature wavelengths (peaks or valleys), and a long time of spectrum scanning and demodulation is required. A rapid demodulation method based on machine learning is proposed in this paper, and a more efficient demodulation of the interference spectra of optical fiber sensors is achieved. It is demonstrated by numerical simulations and experiments that the demodulation performances of the machine learning method are far better than those of the wavelength tracking method when the sampling intervals are sparse. Under a certain accuracy requirement (coefficient of determination larger than 0.9900 and mean square error less than 0.10), the machine learning based method can demodulate the interference spectra with a maximum sampling interval of 8 nm or a minimum wavelength range of 8 nm without feature wavelengths. The demodulation speed can be therefore improved to 400 times that of the wavelength tracking method.

optics,physics, applied
Machine Learning Assisted Fast Demodulation of Large Dynamic Range Dual-Parameter Optical Fiber Sensors

Yong Zhao,Yifan Zhou,Lirong Ren,Jian Zhao,Riqing Lv,Ya-nan Zhang

DOI: https://doi.org/10.1109/jsen.2023.3328362

IF: 4.3

2023-12-16

IEEE Sensors Journal

Abstract:The mutual constraint between sensitivity and measurement range has always been a problem that restricts the development and application of optical fiber interference sensors. We propose a new high-sensitivity tapered dual-parameter optical fiber sensor packaged with polydimethylsiloxane (PDMS) in this article. The use of the multilayer perceptron (MLP) network for demodulation of spectrum without preprocessing improves the constraint relationship between the measurement range and sensitivity of the interferometric sensor, ensuring high sensitivity while expanding the dynamic range. By introducing principal component analysis (PCA) to reduce the dimensionality of the original data, the computational efficiency is improved, and the practicality of the model is improved. As far as we know, this is the first time that PCA and MLP have been used simultaneously to demodulate the spectrum. The detection system has advantages, such as low cost, high sensitivity, easily prepared, fully packaged, and remote real-time monitoring ability. This demodulation method can also be easily extended to demodulation of other multiparameter sensors.

engineering, electrical & electronic,instruments & instrumentation,physics, applied
A Skin-Inspired PDMS Optical Tactile Sensor Driven by a Convolutional Neural Network

Shichao Yue,Minzhi Xu,Zifan Che

DOI: https://doi.org/10.1109/jsen.2024.3355555

IF: 4.3

2024-03-15

IEEE Sensors Journal

Abstract:Tactile sensors play a crucial role in enhancing the integration of automation, robotics, and biomedical equipment, particularly in perceptual functions. Optical fiber-based tactile sensors have gained significance due to their robustness and immunity to electromagnetic interference. However, existing optical fiber-based tactile sensors face limitations related to bio-imitation, scalability, and precise data processing algorithms. This study introduces a novel skin-inspired polydimethylsiloxane (PDMS)-manufactured tactile sensor utilizing a structured light source with low-cost light-emitting diodes and a multimode optical fiber, coupled with tactile information processing through a trained convolutional neural network (CNN). Specklegram images captured from the optical fiber are analyzed for force amplitude and tactile location. The CNN is trained, validated, and tested, achieving accuracies of 99.6%, 99.5%, and 99%, respectively. The tactile sensor demonstrates a spatial resolution of 2 mm and a force-sensing range up to 3 N. The confusion matrix, based on classification results, reveals only three misclassifications out of 315 tests, indicating a mean absolute error (MAE) of 0.95%. The spatial resolution and force-sensing capabilities, coupled with the machine learning approach of the proposed tactile sensor, showcase promising potential for future applications in tactile embodiment.

engineering, electrical & electronic,instruments & instrumentation,physics, applied
Reflective tactile sensor assisted by multimode fiber-based optical coupler and fiber specklegram

Yuezhi Cai,Yan Liu,Guangde Li,Qi Qin,Lezhi Pang,Wenhua Ren,Jie Wei,Muguang Wang

DOI: https://doi.org/10.1016/j.optlastec.2022.109062

2022-12-30

Abstract:A tactile sensor based on the detection of specklegram was proposed using a multimode fiber (MMF)-based optical coupler (OC) with a reflection configuration. Human-like sensations of multiple tactile information are successfully demonstrated. The laser beam was guided by one arm of the MMF-based OC with a coupling ratio of 50:50 and reflected by a soft mirror embedded in a 3D-printed finger-shaped sensor probe. The specklegram interrogated by a camera was obtained at the reflection end of the MMF-based OC. Contact force sensing was firstly realized by six kinds of processing algorithms and the zero-mean normalized cross-correlation coefficient was further used to make an in-depth investigation. Maximal linear measurement range of 30 N was obtained. To evaluate the texture detection ability, the proposed sensor was tested by scanning four 3D-printed texture plates with different periods, and the period obtained by processing the experimental results was highly consistent with that of the texture plates. In addition, the specklegrams from 55 cases when 11 different contact forces were applied at 5 different positions of the sensor probe can be classified with an accuracy of 98 %, which shows the sensor is capable of recognizing the contact position. Furthermore, three kinds of sensor probes with different shore hardness and three different MMF-based OCs were used for comparative analysis.

optics,physics, applied
Demodulation of Fabry–Perot Sensors Using Random Speckles

Qin Liang,Jinchao Tao,Xu Wang,Tianglian Wang,Xinyu Gao,Pengwei Zhou,Ben Xu,Chunliu Zhao,Juan Kang,Le Wang,Chengyu Shen,Dongning Wang,Yi LI

DOI: https://doi.org/10.1364/ol.465212

2022-01-01

Abstract:Random speckles are proposed to demodulate Fabry–Perot (FP) sensors in this study. A piece of multimode fiber is used to interrogate the FP transmission spectrum, and tiny spectral changes lead to significant variations in the generated speckle patterns. In the demonstration experiments, the pressure resolution of 0.001 MPa can be obtained from an open cavity FP sensor based on the convolutional neural network (CNN) demodulation algorithm. It is worth noting that the spectral differences in neighboring orders can be precisely distinguished due to the high sensitivity of speckles. Thus, the fringe-order ambiguity problem is solved and the dynamic measurement range can be greatly improved. The speckle-based demodulation scheme provides a new way to balance resolution, dynamic range, speed, and cost of FP sensors.
Speckle wavemeter based on a multi-core fiber and compressive imaging

Huan Liu,Haole Kong,Jiajun He,Yanqing Qiu,Bangning Mao,Yanlong Meng,Yanghui Li,Juan Kang,Le Wang,Yi Li

DOI: https://doi.org/10.1364/ao.509853

IF: 1.9

2024-01-19

Applied Optics

Abstract:Huan Liu, Haole Kong, Jiajun He, Yanqing Qiu, Bangning Mao, Yanlong Meng, Yanghui Li, Juan Kang, Le Wang, Yi Li Random speckle patterns contain valuable information about the incident light. Researchers have successfully constructed spectrometers and ... [Appl. Opt. 63, 846-852 (2024)]

optics
Magnetic Field Sensing Based on Multimode Fiber Specklegrams

Run-ze Zhu,Sheng-jie Wan,Yi-feng Xiong,Hao-gong Feng,Ye Chen,Yan-qing Lu,Fei Xu,Yanqing Lu

DOI: https://doi.org/10.1109/jlt.2021.3067332

IF: 4.7

2021-06-01

Journal of Lightwave Technology

Abstract:Run-ze Zhu, Sheng-jie Wan, Yi-feng Xiong, Hao-gong Feng, Ye Chen, Yan-qing Lu, Fei Xu<br/>We demonstrate a fiber magnetic field sensor based on multimode fiber specklegrams. The magnetic field is detected by measuring the change ... [J. Lightwave Technol. 39, 3614-3619 (2021)]

engineering, electrical & electronic,optics,telecommunications
The Optimization of Multimode Fiber Speckle Sensor for Microvibration

Haoyang Song,Feiyu Sun,Yunxu Sun,Wei Liu

DOI: https://doi.org/10.1155/2023/3356849

2023-01-01

International Journal of Optics

Abstract:A vibration sensing system with optical fiber speckles is demonstrated and optimized with different optical fiber diameters and speckle statistical algorithms. The types of fiber diameter and material lead to a different performance of fiber specklegram sensor (FSS), which has been experimentally explored. The signal intensity, demodulated from the speckles, is different when using multimode fibers with different diameters. At the same time, the sensing effect of different fibers depends on the speckle statistical algorithms. Accordingly, we use different statistical methods in theory and experiment to analyze the influence of fiber diameter and speckle statistical methods on the sensing performance. A vibration sensing system with optimized performance is achieved by the optimized types of optical fiber and the corresponding optimized algorithms, which are promising for sensing weak vibration, such as detecting.
Fiber-Optic Point-Based Sensor Using Specklegram Measurement

Jiao-Jiao Wang,Shao-Cheng Yan,Ya-Ping Ruan,Fei Xu,Yan-Qing Lu

DOI: https://doi.org/10.3390/s17102429

IF: 3.9

2017-01-01

Sensors

Abstract:Here, we report a fiber-optic point-based sensor to measure temperature and weight based on correlated specklegrams induced by spatial multimode interference. The device is realized simply by splicing a multimode fiber (MMF) to a single-mode fiber (SMF) with a core offset. A series of experiments demonstrates the approximately linear relation between the correlation coefficient and variation. Furthermore, we show the potential applications of the refractive index sensing of our device by disconnecting the splicing point of MMF and SMF. A modification of the algorithm in order to improve the sensitivity of the sensor is also discussed at the end of the paper.
A Machine Learning Specklegram Wavemeter (MaSWave) Based on a Short Section of Multimode Fiber as the Dispersive Element

Ogbole C Inalegwu,Rex E Gerald Ii,Jie Huang

DOI: https://doi.org/10.3390/s23104574

2023-05-09

Abstract:Wavemeters are very important for precise and accurate measurements of both pulses and continuous-wave optical sources. Conventional wavemeters employ gratings, prisms, and other wavelength-sensitive devices in their design. Here, we report a simple and low-cost wavemeter based on a section of multimode fiber (MMF). The concept is to correlate the multimodal interference pattern (i.e., speckle patterns or specklegrams) at the end face of an MMF with the wavelength of the input light source. Through a series of experiments, specklegrams from the end face of an MMF as captured by a CCD camera (acting as a low-cost interrogation unit) were analyzed using a convolutional neural network (CNN) model. The developed machine learning specklegram wavemeter (MaSWave) can accurately map specklegrams of wavelengths up to 1 pm resolution when employing a 0.1 m long MMF. Moreover, the CNN was trained with several categories of image datasets (from 10 nm to 1 pm wavelength shifts). In addition, analysis for different step-index and graded-index MMF types was carried out. The work shows how further robustness to the effects of environmental changes (mainly vibrations and temperature changes) can be achieved at the expense of decreased wavelength shift resolution, by employing a shorter length MMF section (e.g., 0.02 m long MMF). In summary, this work demonstrates how a machine learning model can be used for the analysis of specklegrams in the design of a wavemeter.
Learning to transmit images through optical speckle of a multimode fiber with high fidelity

Pusong Tang,Kanpei Zheng,Weiming Yuan,Tuqiang Pan,Yi Xu,Songnian Fu,Yuncai Wang,Yuwen Qin

DOI: https://doi.org/10.1063/5.0099159

IF: 4

2022-01-01

Applied Physics Letters

Abstract:Multimode fibers provide a unique opportunity for exploring the spatial degrees of freedom for high throughput light transmission. However, the modal dispersion prevents from the straightforward application of multimode fibers for space division multiplexing, such as image transmission. Herein, we propose and experimentally demonstrate a deep neural network termed multimode fiber inverse-scattering net for overcoming the modal dispersion induced scrambling in multimode fibers. Such a network is capable of transmitting grayscale image through the multimode fiber with high fidelity. 256-level grayscale images with 128 x 128 spatial channels encoded in the input wavefront can be retrieved from the output optical speckle patterns, where the average Pearson correlation coefficient and structural similarity index are as large as 0.97 and 0.95, respectively. Our results demonstrate that the proposed deep neural network has an excellent ability for learning the relationship between the input and output optical fields of a multimode fiber, which might facilitate the realization of high throughput space division multiplexing through multimode fibers. Published under an exclusive license by AIP Publishing.
Reflective Optical Tactile Sensor Based on Fiber Specklegram Analysis With the Capability of Contact Position Identification

Yan Liu,Xiaoli Zou,Guangde Li,Qi Qin,Wenhua Ren,Jie Wei,Zhongwei Tan,Muguang Wang

DOI: https://doi.org/10.1109/jlt.2022.3229753

IF: 4.7

2023-04-07

Journal of Lightwave Technology

Abstract:Yan Liu, Xiaoli Zou, Guangde Li, Qi Qin, Wenhua Ren, Jie Wei, Zhongwei Tan, Muguang Wang A reflective tactile sensing scheme based on fiber specklegram analysis is proposed in this study. The specially designed sensor probe with ... [J. Lightwave Technol. 41, 2540-2546 (2023)]

engineering, electrical & electronic,optics,telecommunications
Super-resolution imaging through a multimode fiber: the physical upsampling of speckle-driven

Chuncheng Zhang,Tingting Liu,Zhihua Xie,Yu Wang,Tong Liu,Qian Chen,Xiubao Sui

2023-07-11

Abstract:Following recent advancements in multimode fiber (MMF), miniaturization of imaging endoscopes has proven crucial for minimally invasive surgery in vivo. Recent progress enabled by super-resolution imaging methods with a data-driven deep learning (DL) framework has balanced the relationship between the core size and resolution. However, most of the DL approaches lack attention to the physical properties of the speckle, which is crucial for reconciling the relationship between the magnification of super-resolution imaging and the quality of reconstruction quality. In the paper, we find that the interferometric process of speckle formation is an essential basis for creating DL models with super-resolution imaging. It physically realizes the upsampling of low-resolution (LR) images and enhances the perceptual capabilities of the models. The finding experimentally validates the role played by the physical upsampling of speckle-driven, effectively complementing the lack of information in data-driven. Experimentally, we break the restriction of the poor reconstruction quality at great magnification by inputting the same size of the speckle with the size of the high-resolution (HR) image to the model. The guidance of our research for endoscopic imaging may accelerate the further development of minimally invasive surgery.

Optics,Image and Video Processing,Medical Physics
Fiber-Optic Shape Sensing Using Neural Networks Operating on Multispecklegrams

Caroline G. L. Cao,Bernard Javot,Shreeram Bhattarai,Karin Bierig,Ivan Oreshnikov,Valentin V. Volchkov

DOI: https://doi.org/10.1109/jsen.2024.3430381

IF: 4.3

2024-09-03

IEEE Sensors Journal

Abstract:Application of machine learning techniques on fiber speckle images to infer fiber deformation allows the use of an unmodified multimode fiber to act as a shape sensor. This approach eliminates the need for complex fiber design or construction (e.g., Bragg gratings and time-of-flight). Prior work in shape determination using neural networks trained on a finite number of possible fiber shapes (formulated as a classification task), or trained on a few continuous degrees of freedom, has been limited to reconstruction of fiber shapes only one bend at a time. Furthermore, generalization to shapes that were not used in training is challenging. Our innovative approach improves generalization capabilities, using computer vision-assisted parameterization of the actual fiber shape to provide a ground truth, and multiple specklegrams per fiber shape obtained by controlling the input field. Results from experimenting with several neural network architectures, shape parameterization, number of inputs, and specklegram resolution show that fiber shapes with multiple bends can be accurately predicted. Our approach is able to generalize to new shapes that were not in the training set. This approach of end-to-end training on parameterized ground truth opens new avenues for fiber-optic sensor applications. We publish the datasets used for training and validation, as well as an out-of-distribution (OOD) test set, and encourage interested readers to access these datasets for their own model development.

engineering, electrical & electronic,instruments & instrumentation,physics, applied
High performance few-mode fiber-based light field direction sensing system using deep convolutional neural network: fiber speckle demodulation network (FSDNET)

Ya Wen,Xing Zhao,Zhixiang Jiang,li haoran,Da Li

DOI: https://doi.org/10.1364/oe.524755

IF: 3.8

2024-05-30

Optics Express

Abstract:Ya Wen, Xing Zhao, Zhixiang Jiang, Haoran Li, Da Li Precisely sensing the light field direction information plays the essential role in the fields of three-dimensional (3D) imaging, light ... [Opt. Express 32, 21629-21642 (2024)]

optics

Machine learning enhanced smart tactile specklegram fiber sensor using spatial mode downsampling

Ameliorated Spatial Division Multiplexing System of FBG Sensors Based on Novel Wavelength Detection and Demodulation Technique

Multipoint and Multi-physical Quantity Sensing by Multimode Fiber with Demodulation of Deep Learning

Application of machine learning in optical fiber sensors

Spatially Variant Deblur and Image Enhancement in a Single Multimode Fiber Imaged by Deep Learning

Multimode optical fiber sensors: from conventional to machine learning-assisted

A rapid demodulation method for the interference spectrum of optical fiber sensors based on machine learning

Machine Learning Assisted Fast Demodulation of Large Dynamic Range Dual-Parameter Optical Fiber Sensors

A Skin-Inspired PDMS Optical Tactile Sensor Driven by a Convolutional Neural Network

Reflective tactile sensor assisted by multimode fiber-based optical coupler and fiber specklegram

Demodulation of Fabry–Perot Sensors Using Random Speckles

Speckle wavemeter based on a multi-core fiber and compressive imaging

Magnetic Field Sensing Based on Multimode Fiber Specklegrams

The Optimization of Multimode Fiber Speckle Sensor for Microvibration

Fiber-Optic Point-Based Sensor Using Specklegram Measurement

A Machine Learning Specklegram Wavemeter (MaSWave) Based on a Short Section of Multimode Fiber as the Dispersive Element

Learning to transmit images through optical speckle of a multimode fiber with high fidelity

Reflective Optical Tactile Sensor Based on Fiber Specklegram Analysis With the Capability of Contact Position Identification

Super-resolution imaging through a multimode fiber: the physical upsampling of speckle-driven

Fiber-Optic Shape Sensing Using Neural Networks Operating on Multispecklegrams

High performance few-mode fiber-based light field direction sensing system using deep convolutional neural network: fiber speckle demodulation network (FSDNET)