Outperformance of Linear-regression-based Methods over Deep Learning in Optical Imaging

Shuming Jiao,Yang Gao,Jun Feng,Ting Lei,Xiaocong Yuan

DOI: https://doi.org/10.1364/3d.2020.jw2a.42

2020-01-01

Abstract:Deep learning has been extensively applied in many optical imaging scenarios. But we show linear-regression-based methods may have certain outperformance in two black-box optical imaging problems that were originally solved by deep learning.

What problem does this paper attempt to address?

Does Deep Learning Always Outperform Simple Linear Regression in Optical Imaging?

Shuming Jiao,Yang Gao,Jun Feng,Ting Lei,Xiaocong Yuan

DOI: https://doi.org/10.1364/oe.382319

IF: 3.8

2020-01-01

Optics Express

Abstract:Deep learning has been extensively applied in many optical imaging problems in recent years. Despite the success, the limitations and drawbacks of deep learning in optical imaging have been seldom investigated. In this work, we show that conventional linear-regression-based methods can outperform the previously proposed deep learning approaches for two black-box optical imaging problems in some extent. Deep learning demonstrates its weakness especially when the number of training samples is small. The advantages and disadvantages of linear-regression-based methods and deep learning are analyzed and compared. Since many optical systems are essentially linear, a deep learning network containing many nonlinearity functions sometimes may not be the most suitable option.
4-K-resolution Minimalist Optical System Design Based on Deep Learning.

Dexiao Meng,Yan Zhou,Jian Bai

DOI: https://doi.org/10.1364/ao.510860

IF: 1.9

2023-01-01

Applied Optics

Abstract:In order to simplify optical systems, we propose a high-resolution minimalist optical design method based on deep learning. Unlike most imaging system design work, we combine optical design more closely with image processing algorithms. For optical design, we separately study the impact of different aberrations on computational imaging and then innovatively propose an aberration metric and a spatially micro-variant design method that better meet the needs of image recognition. For image processing, we construct a dataset based on the point spread function (PSF) imaging simulation method. In addition, we use a non-blind deblurring computational imaging method to repair spatially variant aberrations. Finally, we achieve clear imaging at 4 K (5184×3888) using only two spherical lenses and achieve image quality similar to that of complex lenses on the market.
A Comparison of Linear Regression and Deep Learning Model for EVM Estimation in Coherent Optical Systems

Yuchuan Fan,Xiaodan Pang,Aleksejs Udalcovs,Carlos Natalino,Lu Zhang,Sandis Spolitis,Vjaceslavs Bobrovs,Richard Schatz,Xianbin Yu,Marija Furdek,Sergei Popov,Oskars Ozolins

DOI: https://doi.org/10.1109/cleo-pr62338.2022.10432177

2022-01-01

Abstract:We experimentally investigate EVM estimation approaches based on linear regression and deep learning for 28 Gbaud coherent optical systems. We show that the estimation performances are comparable when the modulation format is known.
Deep learning in optical metrology: a review

Chao Zuo,Jiaming Qian,Shijie Feng,Wei Yin,Yixuan Li,Pengfei Fan,Jing Han,Kemao Qian,Qian Chen

DOI: https://doi.org/10.1038/s41377-022-00714-x

2022-02-23

Light: Science and Applications

Abstract:Abstract With the advances in scientific foundations and technological implementations, optical metrology has become versatile problem-solving backbones in manufacturing, fundamental research, and engineering applications, such as quality control, nondestructive testing, experimental mechanics, and biomedicine. In recent years, deep learning, a subfield of machine learning, is emerging as a powerful tool to address problems by learning from data, largely driven by the availability of massive datasets, enhanced computational power, fast data storage, and novel training algorithms for the deep neural network. It is currently promoting increased interests and gaining extensive attention for its utilization in the field of optical metrology. Unlike the traditional “physics-based” approach, deep-learning-enabled optical metrology is a kind of “data-driven” approach, which has already provided numerous alternative solutions to many challenging problems in this field with better performances. In this review, we present an overview of the current status and the latest progress of deep-learning technologies in the field of optical metrology. We first briefly introduce both traditional image-processing algorithms in optical metrology and the basic concepts of deep learning, followed by a comprehensive review of its applications in various optical metrology tasks, such as fringe denoising, phase retrieval, phase unwrapping, subset correlation, and error compensation. The open challenges faced by the current deep-learning approach in optical metrology are then discussed. Finally, the directions for future research are outlined.
Deep Learning Based Computational Imaging: Status, Challenges, and Future

Zuo Chao,Feng Shijie,Zhang Xiangyu,Han Jing,Qian Chen

DOI: https://doi.org/10.3788/aos202040.0111003

2020-01-01

Acta Optica Sinica

Abstract:In recent years, optical imaging techniques have entered into the era of computational optical imaging from the traditional intensity and color imaging. Computational optical imaging, which is based on geometric optics, wave optics, and other theoretical foundations, establishes an accurate forward mathematical model for the whole image formation process of the scene imaged through the optical system and then sampled by the digital detector. Then, the high-quality reconstruction of the image and other high dimensional information, such as phase, spectrum, polarization, light field, coherence, refractive index, and three-dimension profile, which cannot be directly accessed using traditional methods, can be obtained through computational reconstruction method. However, the actual imaging performance of the computational imaging system is also limited by the "accuracy of the forward mathematical model" and "the reliability of inverse reconstruction algorithm ". Besides, the unpredictability of real physical imaging process and the complexity of solving high dimensional ill-posed inverse problems have become the bottleneck of further development of this field. In recent years, the rapid development of artificial intelligence and deep learning for the technology opens a new door for computational optical imaging technology. Unlike " physical driven" model that traditional computational imaging method is based on, computational imaging based on deep learning is a kind of "data-driven" method, which not only solves many problems considered quite challenge to be solved in this field, but also achieves remarkable improvement in information acquisition ability, imaging functions, and key performance indexes of imaging system, such as spatial resolution, temporal resolution, and detection sensitivity. This review first briefly introduces the current status and the latest progress of deep learning technology in the field of computational optical imaging. Then, the main problems and challenges faced by the current deep learning method in computational optical imaging field arc discussed. Finally, the future developments and possible research directions of this field arc prospected.
Deep-Learning-Based Image Reconstruction and Enhancement in Optical Microscopy

Kevin de Haan,Y. Rivenson,Yichen Wu,A. Ozcan

DOI: https://doi.org/10.1109/JPROC.2019.2949575

IF: 20.6

2020-01-01

Proceedings of the IEEE

Abstract:In recent years, deep learning has been shown to be one of the leading machine learning techniques for a wide variety of inference tasks. In addition to its mainstream applications, such as classification, it has created transformative opportunities for image reconstruction and enhancement in optical microscopy. Some of these emerging applications of deep learning range from image transformations between microscopic imaging systems to adding new capabilities to existing imaging techniques, as well as solving various inverse problems based on microscopy image data. Deep learning is helping us move toward data-driven instrument designs that blend microscopy and computing to achieve what neither can do alone. This article provides an overview of some of the recent work using deep neural networks to advance computational microscopy and sensing systems, also covering their current and future biomedical applications.

Physics,Engineering,Computer Science
Surpassing the Standard Quantum Limit of Optical Imaging Via Deep Learning

Miao Cai,Zhi-Xiang Li,Hao-Dong Wu,Ya-Ping Ruan,Lei Tang,Jiang -Shan Tang,Ming -Yuan Chen,Han Zhang,Ke-Yu Xia,Min Xiao,Yan-Qing Lu

DOI: https://doi.org/10.3788/col202321.082701

IF: 2.56

2023-01-01

Chinese Optics Letters

Abstract:The sensitivity of optical measurement is ultimately constrained by the shot noise to the standard quantum limit.It has become a common concept that beating this limit requires quantum resources.A deep-learning neural network free of quantum principle has the capability of removing classical noise from images,but it is unclear in reducing quantum noise.In a coincidence-imaging experiment,we show that quantum-resource-free deep learning can be exploited to surpass the standard quantum limit via the photon-number-dependent nonlinear feedback during training.Using an effective classical light with photon flux of about 9 × 104 photons per second,our deep-learning-based scheme achieves a 14 dB improvement in signal-to-noise ratio with respect to the standard quantum limit.
Deep learning in macroscopic diffuse optical imaging

Jason T. Smith,Marien Ochoa,Denzel Faulkner,Grant Haskins,Xavier Intes

DOI: https://doi.org/10.1117/1.jbo.27.2.020901

IF: 3.758

2022-02-25

Journal of Biomedical Optics

Abstract:SIGNIFICANCE: Biomedical optics system design, image formation, and image analysis have primarily been guided by classical physical modeling and signal processing methodologies. Recently, however, deep learning (DL) has become a major paradigm in computational modeling and has demonstrated utility in numerous scientific domains and various forms of data analysis.AIM: We aim to comprehensively review the use of DL applied to macroscopic diffuse optical imaging (DOI).APPROACH: First, we provide a layman introduction to DL. Then, the review summarizes current DL work in some of the most active areas of this field, including optical properties retrieval, fluorescence lifetime imaging, and diffuse optical tomography.RESULTS: The advantages of using DL for DOI versus conventional inverse solvers cited in the literature reviewed herein are numerous. These include, among others, a decrease in analysis time (often by many orders of magnitude), increased quantitative reconstruction quality, robustness to noise, and the unique capability to learn complex end-to-end relationships.CONCLUSIONS: The heavily validated capability of DL's use across a wide range of complex inverse solving methodologies has enormous potential to bring novel DOI modalities, otherwise deemed impractical for clinical translation, to the patient's bedside.

radiology, nuclear medicine & medical imaging,biochemical research methods,optics
Deep learning-based image processing in optical microscopy

Sindhoora Kaniyala Melanthota,Dharshini Gopal,Shweta Chakrabarti,Anirudh Ameya Kashyap,Raghu Radhakrishnan,Nirmal Mazumder

DOI: https://doi.org/10.1007/s12551-022-00949-3

2022-04-01

Biophysical Reviews

Abstract:Abstract Optical microscopy has emerged as a key driver of fundamental research since it provides the ability to probe into imperceptible structures in the biomedical world. For the detailed investigation of samples, a high-resolution image with enhanced contrast and minimal damage is preferred. To achieve this, an automated image analysis method is preferable over manual analysis in terms of both speed of acquisition and reduced error accumulation. In this regard, deep learning (DL)-based image processing can be highly beneficial. The review summarises and critiques the use of DL in image processing for the data collected using various optical microscopic techniques. In tandem with optical microscopy, DL has already found applications in various problems related to image classification and segmentation. It has also performed well in enhancing image resolution in smartphone-based microscopy, which in turn enablse crucial medical assistance in remote places. Graphical abstract
Fast Correlated-Photon Imaging Enhanced By Deep Learning

Zhan-Ming Li,Shi-Bao Wu,Jun Gao,Heng Zhou,Zeng-Quan Yan,Ruo-Jing Ren,Si-Yuan Yin,Xian-Min Jin

DOI: https://doi.org/10.1364/OPTICA.408843

IF: 10.4

2021-01-01

Optica

Abstract:Quantum imaging using photon pairs with strong quantum correlations has been harnessed to bring quantum advantages to various fields from biological imaging to range finding. Such inherent non-classical properties support the extraction of more valid signals to build photon-limited images, even in low-light conditions where the shot noise becomes dominant as light decreases to a single-photon level. Numerical optimization algorithms are possible but require thousands of photon-sparse frames, and they are thus unavailable in real time. We demonstrate fast correlated-photon imaging enhanced by deep learning as an intelligent computational strategy to discover a deeper structure in big data. Our work verifies that a convolutional neural network can efficiently solve inverse imaging problems associated with strong shot noise and background noise (electronic noise, scattered light). Our results show that we can overcome limitations due to the trade-off between imaging speed and image quality by pushing the low-light imaging technique to the single-photon level in real time, which enables deep-learning-enhanced quantum imaging for real-life applications. (C) 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
Development and quantitative assessment of deep learning-based image enhancement for optical coherence tomography

Xinyu Zhao,Bin Lv,Lihui Meng,Xia Zhou,Dongyue Wang,Wenfei Zhang,Erqian Wang,Chuanfeng Lv,Guotong Xie,Youxin Chen

DOI: https://doi.org/10.1186/s12886-022-02299-w

IF: 2.086

2022-03-26

BMC Ophthalmology

Abstract:Abstract Purpose To develop a deep learning-based framework to improve the image quality of optical coherence tomography (OCT) and evaluate its image enhancement effect with the traditional image averaging method from a clinical perspective. Methods 359 normal eyes and 456 eyes with various retinal conditions were included. A deep learning framework with high-resolution representation was developed to achieve image quality enhancement for OCT images. The quantitative comparisons, including expert subjective scores from ophthalmologists and three objective metrics of image quality (structural similarity index measure (SSIM), peak signal-to-noise ratio (PSNR) and contrast-to-noise ratio (CNR)), were performed between deep learning method and traditional image averaging. Results With the increase of frame count from 1 to 20, our deep learning method always obtained higher SSIM and PSNR values than the image averaging method while importing the same number of frames. When we selected 5 frames as inputs, the local objective assessment with CNR illustrated that the deep learning method had more obvious tissue contrast enhancement than averaging method. The subjective scores of image quality were all highest in our deep learning method, both for normal retinal structure and various retinal lesions. All the objective and subjective indicators had significant statistical differences ( P < 0.05). Conclusion Compared to traditional image averaging methods, our proposed deep learning enhancement framework can achieve a reasonable trade-off between image quality and scanning times, reducing the number of repeated scans.

ophthalmology
Deep Learning Optical-Sectioning Method

Xiaoyu Zhang,Yifan Chen,Kefu Ning,Can Zhou,Yutong Han,Hui Gong,Jing Yuan

DOI: https://doi.org/10.1364/oe.26.030762

IF: 3.8

2018-01-01

Optics Express

Abstract:Current optical-sectioning methods require complex optical system or considerable computation time to improve imaging quality. Here we propose a deep learning-based method for optical sectioning of wide-field images. This method only needs one pair of contrast images for training to facilitate reconstruction of an optically sectioned image. The removal effect of background information and resolution that is achievable with our technique is similar to traditional optical-sectioning methods, but offers lower noise levels and a higher imaging depth. Moreover, reconstruction speed can be optimized to 14 Hz. This cost-effective and convenient method enables high-throughput optical sectioning techniques to be developed.
Deep learning in biomedical optics

Lei Tian,Brady Hunt,Muyinatu A. Lediju Bell,Ji Yi,Jason T. Smith,Marien Ochoa,Xavier Intes,Nicholas J. Durr

DOI: https://doi.org/10.1002/lsm.23414

2021-05-24

Abstract:This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each of these domains, we summarize how deep learning has been applied and highlight methods by which deep learning can enable new capabilities for optics in medicine. Challenges and opportunities to improve translation and adoption of deep learning in biomedical optics are also summarized.

Optics,Biological Physics
Deep Neural Networks for Computational Optical Form Measurements

Lara Hoffmann,Clemens Elster

DOI: https://doi.org/10.5194/jsss-9-301-2020

2020-07-01

Abstract:Deep neural networks have been successfully applied in many different fields like computational imaging, medical healthcare, signal processing, or autonomous driving. In a proof-of-principle study, we demonstrate that computational optical form measurement can also benefit from deep learning. A data-driven machine learning approach is explored to solve an inverse problem in the accurate measurement of optical surfaces. The approach is developed and tested using virtual measurements with known ground truth.

Image and Video Processing,Machine Learning,Instrumentation and Detectors
Deep Learning for Optical Sensor Applications: A Review

Nagi H. Al-Ashwal,Khaled A. M. Al Soufy,Mohga E. Hamza,Mohamed A. Swillam

DOI: https://doi.org/10.3390/s23146486

IF: 3.9

2023-07-19

Sensors

Abstract:Over the past decade, deep learning (DL) has been applied in a large number of optical sensors applications. DL algorithms can improve the accuracy and reduce the noise level in optical sensors. Optical sensors are considered as a promising technology for modern intelligent sensing platforms. These sensors are widely used in process monitoring, quality prediction, pollution, defence, security, and many other applications. However, they suffer major challenges such as the large generated datasets and low processing speeds for these data, including the high cost of these sensors. These challenges can be mitigated by integrating DL systems with optical sensor technologies. This paper presents recent studies integrating DL algorithms with optical sensor applications. This paper also highlights several directions for DL algorithms that promise a considerable impact on use for optical sensor applications. Moreover, this study provides new directions for the future development of related research.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Deep learning in optics and photonics for statistical inference, computing, and inverse design

Deniz Mengu,Md Sadman Sakib Rahman,Yi Luo,Onur Kulce,Aydogan Ozcan,jingxi li

DOI: https://doi.org/10.1364/aop.450345

IF: 24.75

2022-04-07

Advances in Optics and Photonics

Abstract:Deep learning has been revolutionizing information processing in many fields of science and engineering owing to the massively growing amounts of data and the advances in deep neural network architectures. As these neural networks are expanding their capabilities toward achieving state-of-the-art solutions for demanding statistical inference tasks in various applications, there appears to be a global need for low-power, scalable, and fast computing hardware beyond what existing electronic systems can offer. Optical computing might potentially address some of these needs with its inherent parallelism, power efficiency, and high speed. Recent advances in optical materials, fabrication, and optimization techniques have significantly enriched the design capabilities in optics and photonics, leading to various successful demonstrations of guided-wave and free-space computing hardware for accelerating machine learning tasks using light. In addition to statistical inference and computing, deep learning has also fundamentally affected the field of inverse optical/photonic design. The approximation power of deep neural networks has been utilized to develop optics/photonics systems with unique capabilities, all the way from nanoantenna design to end-to-end optimization of computational imaging and sensing systems. In this review, we attempt to provide a broad overview of the current state of this emerging symbiotic relationship between deep learning and optics/photonics.

optics
A comparative evaluation of deep learning approaches for ophthalmology

Glenn Linde,Waldir Rodrigues de Souza Jr,Renoh Chalakkal,Helen V. Danesh-Meyer,Ben O'Keeffe,Sheng Chiong Hong

DOI: https://doi.org/10.1038/s41598-024-72752-x

IF: 4.6

2024-09-20

Scientific Reports

Abstract:There is a growing number of publicly available ophthalmic imaging datasets and open-source code for Machine Learning algorithms. This allows ophthalmic researchers and practitioners to independently perform various deep-learning tasks. With the advancement in artificial intelligence (AI) and in the field of imaging, the choice of the most appropriate AI architecture for different tasks will vary greatly. The best-performing AI-dataset combination will depend on the specific problem that needs to be solved and the type of data available. The article discusses different machine learning models and deep learning architectures currently used for various ophthalmic imaging modalities and for different machine learning tasks. It also proposes the most appropriate models based on accuracy and other important factors such as training time, the ability to deploy the model on clinical devices/smartphones, heatmaps that enhance the self-explanatory nature of classification decisions, and the ability to train/adapt on small image datasets to determine if further data collection is worthwhile. The article extensively reviews the existing state-of-the-art AI methods focused on useful machine-learning applications for ophthalmology. It estimates their performance and viability through training and evaluating architectures with different public and private image datasets of different modalities, such as full-color retinal images, OCT images, and 3D OCT scans. The article is expected to benefit the readers by enriching their knowledge of artificial intelligence applied to ophthalmology.

multidisciplinary sciences
Optical Machine Learning with Single-pixel Imaging

Shuming Jiao,Jun Feng,Yang Gao,Ting Lei,Zhenwei Xie,Xiaocong Yuan

DOI: https://doi.org/10.1364/3d.2020.jw2a.43

2020-01-01

Abstract:An optical learning machine is designed based on single-pixel imaging, which can perform pattern recognition full-optically without object image reconstruction. It has advantages of incoherent illumination, low experimental complexity and being programmable.
Deep Learning Microscopy

Yair Rivenson,Zoltan Gorocs,Harun Gunaydin,Yibo Zhang,Hongda Wang,Aydogan Ozcan

DOI: https://doi.org/10.1364/OPTICA.4.001437

2017-05-13

Abstract:We demonstrate that a deep neural network can significantly improve optical microscopy, enhancing its spatial resolution over a large field-of-view and depth-of-field. After its training, the only input to this network is an image acquired using a regular optical microscope, without any changes to its design. We blindly tested this deep learning approach using various tissue samples that are imaged with low-resolution and wide-field systems, where the network rapidly outputs an image with remarkably better resolution, matching the performance of higher numerical aperture lenses, also significantly surpassing their limited field-of-view and depth-of-field. These results are transformative for various fields that use microscopy tools, including e.g., life sciences, where optical microscopy is considered as one of the most widely used and deployed techniques. Beyond such applications, our presented approach is broadly applicable to other imaging modalities, also spanning different parts of the electromagnetic spectrum, and can be used to design computational imagers that get better and better as they continue to image specimen and establish new transformations among different modes of imaging.

Machine Learning,Computer Vision and Pattern Recognition,Optics
Deep Learning with Passive Optical Nonlinear Mapping

Fei Xia,Kyungduk Kim,Yaniv Eliezer,Liam Shaughnessy,Sylvain Gigan,Hui Cao

2023-07-18

Abstract:Deep learning has fundamentally transformed artificial intelligence, but the ever-increasing complexity in deep learning models calls for specialized hardware accelerators. Optical accelerators can potentially offer enhanced performance, scalability, and energy efficiency. However, achieving nonlinear mapping, a critical component of neural networks, remains challenging optically. Here, we introduce a design that leverages multiple scattering in a reverberating cavity to passively induce optical nonlinear random mapping, without the need for additional laser power. A key advantage emerging from our work is that we show we can perform optical data compression, facilitated by multiple scattering in the cavity, to efficiently compress and retain vital information while also decreasing data dimensionality. This allows rapid optical information processing and generation of low dimensional mixtures of highly nonlinear features. These are particularly useful for applications demanding high-speed analysis and responses such as in edge computing devices. Utilizing rapid optical information processing capabilities, our optical platforms could potentially offer more efficient and real-time processing solutions for a broad range of applications. We demonstrate the efficacy of our design in improving computational performance across tasks, including classification, image reconstruction, key-point detection, and object detection, all achieved through optical data compression combined with a digital decoder. Notably, we observed high performance, at an extreme compression ratio, for real-time pedestrian detection. Our findings pave the way for novel algorithms and architectural designs for optical computing.

Optics,Disordered Systems and Neural Networks,Emerging Technologies,Machine Learning,Applied Physics

Outperformance of Linear-regression-based Methods over Deep Learning in Optical Imaging

Does Deep Learning Always Outperform Simple Linear Regression in Optical Imaging?

4-K-resolution Minimalist Optical System Design Based on Deep Learning.

A Comparison of Linear Regression and Deep Learning Model for EVM Estimation in Coherent Optical Systems

Deep learning in optical metrology: a review

Deep Learning Based Computational Imaging: Status, Challenges, and Future

Deep-Learning-Based Image Reconstruction and Enhancement in Optical Microscopy

Surpassing the Standard Quantum Limit of Optical Imaging Via Deep Learning

Deep learning in macroscopic diffuse optical imaging

Deep learning-based image processing in optical microscopy

Fast Correlated-Photon Imaging Enhanced By Deep Learning

Development and quantitative assessment of deep learning-based image enhancement for optical coherence tomography

Deep Learning Optical-Sectioning Method

Deep learning in biomedical optics

Deep Neural Networks for Computational Optical Form Measurements

Deep Learning for Optical Sensor Applications: A Review

Deep learning in optics and photonics for statistical inference, computing, and inverse design

A comparative evaluation of deep learning approaches for ophthalmology

Optical Machine Learning with Single-pixel Imaging

Deep Learning Microscopy

Deep Learning with Passive Optical Nonlinear Mapping