Computational Imaging: the Next Revolution for Biophotonics and Biomedicine

An Pan,Baoli Yao,Chao Zuo,Fei Liu,Jiamiao Yang,Liangcai Cao

DOI: https://doi.org/10.3390/cells13050433

IF: 6

2024-01-01

Cells

Abstract:This Editorial is the preface for the topical collection of “Computational Imaging for Biophotonics and Biomedicine”, which collates the 12 contributions listed in Table 1 [...]

What problem does this paper attempt to address?

Integrative Approaches in Computational Biomedical Imaging

Huafeng Liu,Pengcheng Shi,Yunmei Chen

DOI: https://doi.org/10.1155/2012/162892

IF: 2.809

2012-01-01

Computational and Mathematical Methods in Medicine

Abstract:Increasingly wider availability of biomedical imaging modalities, such as X-ray computed tomography (CT), magnetic resonance imaging (MRI), ultrasonic imaging, positron emission tomography (PET), single-photon emission computed tomography (SPECT), and optical imaging, have led to significant progresses in both biomedical research and clinical practice. Advancements in imaging technology and image informatics have facilitated and enabled quantitative understanding of the biological structures, functional organisms, and pathological mechanisms. This special issue presents 14 latest research contributions in computational biomedical imaging. The focuses have been on the computational principles, strategies, methods, and algorithms that create, extract, and understand imaging data of biomedical significance. The first group of papers has showcased the development and applications of rigorous optimization techniques in biomedical imaging. The work based on dual unscented Kalman filtering provides a reliable and efficient platform to understand the BOLD signals from fMRI. In the other paper, the robust discrete particle swarm optimization algorithm is adopted to solve the branch-cut phase unwrapping problem of MRI data. Without the need for a regularization penalty term, a semigreed method for bioluminescence tomography reconstruction is fast and stable for small animal imaging. The second group of papers has addressed image segmentation and tissue characterization problems, ever popular among image analysis researchers. The work of L. Li et al. applies machine learning techniques to characterize lymph node metastasis in gastric cancer from gemstone spectral imaging with much improved outcomes, while G. V. Sanchez. Ferrero et al. make new advances in ultrasonic speckle characterization using generalized Gama distribution and generalized Gama mixture model. In recent years, physically meaningful modeling and simulation have become integral components for model-constrained imaging and model-based image understanding. Thus, the third group of papers has continued the stride in this direction. The propagation of myocardial electrical activation is simulated using a monodomain model, without the need of explicit mesh constraints. In similar spirit, one work performs forward modeling of 3D optical molecular imaging with simplified spherical harmonics approximation using extended finite element method. The final group of papers has highlighted some new imaging modalities and their potential use in clinical practice. The further development of ultrasound-based quantitative imaging of Young's modulus with high accuracy is presented in this issue. And the novel Diffusion Kurtosis Imaging shows that more detailed classification of brain tissues is clinical significance. We appreciate the many high quality submissions to this special issue from leading researchers in computational biomedical imaging. Because of space limitation, we can only accept a small number of most exciting works. We sincerely hope that the remaining interesting research outcomes will be soon published in other venues.
Trans-scale Optical Endoscopy Imaging Technology

Wang Zi-chuan,Zhang Wei,Guo Fei,Jia Zhi-qiang,Wang Li-qiang,Dong Wen-fei,Yang Qing

DOI: https://doi.org/10.37188/co.2022-0078

IF: 1.1

2022-01-01

Chinese Optics

Abstract:Due to the advantages of high resolution, multi-scale, multi-dimension, low radiation and easy to integrate, optical imaging technology plays an important role in biomedical field. In the field of endoscopy, how to obtain, process and visualize the endoscopic image information is the core of the problem what optical imaging technology need to solve. The obtaining of trans-scale endoscopic image of patients in the medical clinical is more advantageous to the surgeon for the diagnosis of patients and can improve in accuracy of the operation. The review starts with the application of trans-scale optical imaging technology in the field of endoscopy, focusing on the different optical systems to obtain trans-scale images in clinical endoscopy, including trans-scale optical zoom system, multi-channel imaging system, fiber-scanning imaging system, and expounds its progress and future trends.
Computational Photography：Keys, Methods and Applications

Jin-Li SUO,Ye-Bin LIU,Xiang-Yang JI,Qiong-Hai DAI

DOI: https://doi.org/10.16383/j.aas.2015.c130855

2015-01-01

Abstract:Current imaging mechanisms and systems cannot capture su?cient visual information of target ob jects/scenes in many fields, such as computer vision, graphics, signal processing, digital image processing, applied optics, etc. To address these challenges, computational photography has proposed new imaging mechanisms and corresponding reconstruction methods that bring the visual information processing forward to the acquisition process and largely raise the degree of freedom on information optimization. The computational acquisition approaches are able to breakthrough the bottlenecks in dimension, scale, and resolution, and thus can observe the scenes that cannot be captured clearly by traditional imaging systems. This review focuses on three main aspects of computational photography — strategy, approach, and target —and attempts to familiarize the readers with the studies in this field.
Optical Brain Imaging: A Powerful Tool for Neuroscience

Xinpei Zhu,Yanfang Xia,Xuecen Wang,Ke Si,Wei Gong

DOI: https://doi.org/10.1007/s12264-016-0053-6

2017-01-01

Abstract:As the control center of organisms,the brain remains little understood due to its complexity. Taking advantage of imaging methods,scientists have found an accessible approach to unraveling the mystery of neuroscience. Among these methods,optical imaging techniques are widely used due to their high molecular specificity and single-molecule sensitivity. Here,we overview several optical imaging techniques in neuroscience of recent years,including brain clearing,the micro-optical sectioning tomography system,and deep tissue imaging.
Emerging theories and technologies on computational imaging

Xue-mei Hu,Jia-min Wu,Jin-li Suo,Qiong-hai Dai

DOI: https://doi.org/10.1631/fitee.1700211

2017-09-01

Abstract:Computational imaging describes the whole imaging process from the perspective of light transport and information transmission, features traditional optical computing capabilities, and assists in breaking through the limitations of visual information recording. Progress in computational imaging promotes the development of diverse basic and applied disciplines. In this review, we provide an overview of the fundamental principles and methods in computational imaging, the history of this field, and the important roles that it plays in the development of science. We review the most recent and promising advances in computational imaging, from the perspective of different dimensions of visual signals, including spatial dimension, temporal dimension, angular dimension, spectral dimension, and phase. We also discuss some topics worth studying for future developments in computational imaging.

engineering, electrical & electronic,computer science, information systems, software engineering
Recent Advances in Computational Photography

Dai Qionghai,Wu Jiamin,Fan Jingtao,Xu Feng,Cao Xun

DOI: https://doi.org/10.1049/cje.2018.10.006

IF: 1.019

2019-01-01

Chinese Journal of Electronics

Abstract:Traditional photography focuses on the optimization of lenses for a perfect imaging system. However, with the great developments of computational resources and optical modulation devices, we can achieve more powerful imaging abilities with concise optics. Computational photography is such an emerging interdisciplinary field by incorporating computational strategy in traditional imaging system to break the limitations in various dimensions such as spectrum, time and space. Recent advances in different aspects have aroused great interests and introduced tremendous applications in biology, material science and computer vision.
An Overview of Computational Photography

JinLi Suo,XiangYang Ji,QiongHai Dai

DOI: https://doi.org/10.1007/s11432-012-4587-6

2012-01-01

Science China Information Sciences

Abstract:Computational photography is an emerging multidisciplinary field. Over the last two decades, it has integrated studies across computer vision, computer graphics, signal processing, applied optics and related disciplines. Researchers are exploring new ways to break through the limitations of traditional digital imaging for the benefit of photographers, vision and graphics researchers, and image processing programmers. Thanks to much effort in various associated fields, the large variety of issues related to these new methods of photography are described and discussed extensively in this paper. To give the reader the full picture of the voluminous literature related to computational photography, this paper briefly reviews the wide range of topics in this new field, covering a number of different aspects, including: (i) the various elements of computational imaging systems and new sampling and reconstruction mechanisms; (ii) the different image properties which benefit from computational photography, e.g. depth of field, dynamic range; and (iii) the sampling subspaces of visual scenes in the real world. Based on this systematic review of the previous and ongoing work in this field, we also discuss some open issues and potential new directions in computational photography. This paper aims to help the reader get to know this new field, including its history, ultimate goals, hot topics, research methodologies, and future directions, and thus build a foundation for further research and related developments.
Computational Optical Coherence Tomography [Invited]

Yuan Zhi Liu,Fredrick A. South,Yang Xu,P. Scott Carney,Stephen A. Boppart

DOI: https://doi.org/10.1364/boe.8.001549

IF: 3.4

2017-01-01

Biomedical Optics Express

Abstract:Optical coherence tomography (OCT) has become an important imaging modality with numerous biomedical applications. Challenges in high-speed, high-resolution, volumetric OCT imaging include managing dispersion, the trade-off between transverse resolution and depth-of-field, and correcting optical aberrations that are present in both the system and sample. Physics-based computational imaging techniques have proven to provide solutions to these limitations. This review aims to outline these computational imaging techniques within a general mathematical framework, summarize the historical progress, highlight the state-of-the-art achievements, and discuss the present challenges.
Editorial Introduction to JSTQE Special Issue on Biophotonics (2023)

Qiyin Fang,Charles Kervrann,David Nolte,Oxana Semyachkina-Glushkovskaya,Shuo Tang,Yuji Matsuura,Zachary Smith

DOI: https://doi.org/10.1109/jstqe.2023.3287745

IF: 4.9

2023-01-01

IEEE Journal of Selected Topics in Quantum Electronics

Abstract:Biophotonics is a research and technology field that sits at the interface of photonics and biomedical applications, which includes clinical diagnosis, therapeutics, biotechnology, and the environment. The integration of optics and biology has been one of the most active technology areas in the past three decades, not only in curiosity-driven research but also in terms of how these technologies are translated into practice with visible impacts within a relatively short time. A prominent example is the use of lasers and photonics technologies in ophthalmology applications such as laser vision correction, artificial lenses in cataract surgery, and retinal detachment repairs. The adoption of photonics technologies in medicine is significantly faster than other medical devices in a relatively cautious and heavily regulated area, mostly owing to its natural compatibility with minimally invasive procedures. Biophotonics is an interdisciplinary and multidisciplinary area at the same time. Even for a specific application, e.g., laser vision correction, many different photonics technologies may be used in combination: UV laser ablation in cornea reshaping, ultrafast laser ablation for cap cutting, image processing and modeling for procedure planning and design, and real time eye movement tracking. Many optical modalities do not require direct contact with the target, which allows several methods to be multiplexed, e.g., to provide additional sources of diagnostic contrasts or real-time feedback and guidance for therapy.
High Contrast 3-D Optical Bioimaging Using Molecular and Nanoprobes Optically Responsive to IR Light

Jun Qian,Zhe Feng,Xiaoxiao Fan,Andrey Kuzmin,Anderson S. L. Gomes,Paras N. Prasad

DOI: https://doi.org/10.1016/j.physrep.2022.02.004

IF: 30.51

2022-01-01

Physics Reports

Abstract:Biophotonics is a convergent field which integrates biological physics, neurosciences, optical physics, and nanoscience for optical diagnostics, bioimaging and light activated therapies, with no tissue invasion and free of ionizing radiation. It is expected to play an important role in advancing the emerging field of molecular medicine by yielding detailed information on structures and dynamics at subcellular levels. This review, truly interdisciplinary and unique in its scope, highlights the integrated roles of the above-named disciplines for advancing IR bioimaging, aiming to attract a broad cross-section of readership of Physics Reports, and is written with multinational authorship to bring out a global perspective. A unique feature of this review is its comprehensive coverage of a multitude of the various optical bioimaging modalities, their physics principles, their relative merits, and their complementarity. Specifically covered themes are fluorescence imaging, nonlinear optical imaging, photoacoustic imaging, optical coherence tomography (OCT) and hyperspectral imaging. The review first discusses light propagation and bio-optics of processes of light absorption, scattering and transmission in biological media. It describes the various biological spectral windows of maximum optical transparencies which are suitable for optical imaging. Then, we introduce the different optical imaging modalities and their features in bioimaging both in vitro and in vivo. The role of molecular and nano-probes to serve as contrast agents for different modalities of imaging is described. A unique aspect also discussed in this review is the physics of manipulation of excitation dynamics that can make them selectively nanoemitters for fluorescence imaging; nonlinear optical probes for imaging; nano-photothermal heaters and nanothermometry for photoacoustic imaging; and nanoscatterers for OCT. As selected emerging applications, we describe imaging-guided theranostics as well as applications in novel areas of neurophotonics and nanodentistry. Finally, the review concludes by presenting current challenges and our views on future opportunities aimed to stimulate further research in this field of high societal impact.(C) 2022 Published by Elsevier B.V.
Embodied Computational Imaging: a New Paradigm for Observing and Analyzing Spatiotemporally Ultrasensitive Phenomena at Multiple Scales

Baoquan Chen,Zhouchen Lin,Peng Xi,Yebin Liu,Xiaodian Chen

DOI: https://doi.org/10.1007/s11432-024-4121-0

2024-01-01

Science China Information Sciences

Abstract:>Imaging is the foundational tool for scientists to observe and understand the world. It has served as the basis for scientific discoveries over human history. In recent years, with the rapid advancement of artificial intelligence and computing technologies, a new interdisciplinary field, computational imaging, has emerged and developed. By jointly optimizing imaging hardware and processing algorithms, computational imaging continuously breaks through the physical limits of traditional imaging systems in aspects such as resolution,dimension, contrast, and throughput.
IEEE Access Special Section Editorial: Biologically Inspired Image Processing Challenges and Future Directions

Jiachen Yang,Qinggang Meng,Maurizio Murroni,Shiqi Wang,Feng Shao

DOI: https://doi.org/10.1109/access.2020.3015372

IF: 3.9

2020-01-01

IEEE Access

Abstract:Human kind is exposed to large amounts of data. According to statistics, more than 80% of information received by humans comes from the visual system. Therefore, image information processing is not only an important research topic but also a challenging task. The unique information processing mechanism of the human visual system provides it with fast, accurate, and efficient image processing capabilities. At present, many advanced image analysis and processing techniques have been widely used in image communication, geographic information systems, medical image analysis, and virtual reality. However, there is still a large gap between these technologies and the human visual system. Therefore, building an image system research mechanism based on the biological vision system is an attractive but difficult target. Although it is a challenge, it can also be considered as an opportunity which utilizes biologically inspired ideas. Meanwhile, through the integration of neural biology, biological perception mechanisms, and computer science and mathematical science, related research can bridge biological vision and computer vision. Finally, the biologically inspired image analysis and processing system is expected to be built on the basis of further consideration of the learning mechanism of the human brain.
Deep Learning Based Computational Imaging: Status, Challenges, and Future

Zuo Chao,Feng Shijie,Zhang Xiangyu,Han Jing,Qian Chen

DOI: https://doi.org/10.3788/aos202040.0111003

2020-01-01

Acta Optica Sinica

Abstract:In recent years, optical imaging techniques have entered into the era of computational optical imaging from the traditional intensity and color imaging. Computational optical imaging, which is based on geometric optics, wave optics, and other theoretical foundations, establishes an accurate forward mathematical model for the whole image formation process of the scene imaged through the optical system and then sampled by the digital detector. Then, the high-quality reconstruction of the image and other high dimensional information, such as phase, spectrum, polarization, light field, coherence, refractive index, and three-dimension profile, which cannot be directly accessed using traditional methods, can be obtained through computational reconstruction method. However, the actual imaging performance of the computational imaging system is also limited by the "accuracy of the forward mathematical model" and "the reliability of inverse reconstruction algorithm ". Besides, the unpredictability of real physical imaging process and the complexity of solving high dimensional ill-posed inverse problems have become the bottleneck of further development of this field. In recent years, the rapid development of artificial intelligence and deep learning for the technology opens a new door for computational optical imaging technology. Unlike " physical driven" model that traditional computational imaging method is based on, computational imaging based on deep learning is a kind of "data-driven" method, which not only solves many problems considered quite challenge to be solved in this field, but also achieves remarkable improvement in information acquisition ability, imaging functions, and key performance indexes of imaging system, such as spatial resolution, temporal resolution, and detection sensitivity. This review first briefly introduces the current status and the latest progress of deep learning technology in the field of computational optical imaging. Then, the main problems and challenges faced by the current deep learning method in computational optical imaging field arc discussed. Finally, the future developments and possible research directions of this field arc prospected.
Research Advances in Simple and Compact Optical Imaging Techniques

Liu You-Hai,Qin Tian-Xiang,Wang Ying-Ce,Kang Xing-Wang,Liu Jun,Wu Jia-Chen,Cao Liang-Cai

DOI: https://doi.org/10.7498/aps.72.20230092

IF: 0.906

2023-01-01

Acta Physica Sinica

Abstract:Computational imaging enables optical imaging systems to acquire more information with miniaturized setups. Computational imaging can avoid the object-image conjugate limitation of the imaging system, and introduce encoding and decoding processes based on physical optics to achieve more efficient information transmission. It can simultaneously increase the amount of information and reduce the complexity of the system, thereby paving the way for miniaturizing imaging systems. Based on computational imaging, the simple and compact optical imaging techniques are developed, which is also called simple optics. To develop miniaturized optical imaging elements and integrated systems, simple optics utilizes the joint design of optical system and image processing algorithms, thereby realizing high-quality imaging that is comparable to complex optical systems. The imaging systems are of small-size, low-weight, and low-power consumption. With the development of micro-nano manufacturing, the optical elements have evolved from a single lens or a few lenses, to flat/planar optical elements, such as diffractive optical elements and metasurface optical elements. As a result, various lensless and metalens imaging systems have emerged. Owing to the introduction of encoding process and decoding process, an optical imaging model is developed to represent the relationship between the target object and the acquired signal, from which the computational reconstruction is used to restore the image. In the image restoration part, the algorithms are discussed in three categories, i.e. the classic algorithm, the model-based optimization iterative algorithm, and the deep learning (neural network) algorithm. Besides, the end-to-end optimization is highlighted because it introduces a new frame to minimize the complexity of optical system. In this review, the imaging techniques realized by simple optics are also discussed, such as depth imaging, high-resolution and super-resolution imaging, large field of view imaging, and extended depth of field imaging, as well as their important roles in developing consumer electronics, unmanned driving, machine vision, security monitoring, biomedical devices and metaverse. Last but not least, the challenges and future developments are prospected.
Computational Optical Sectioning by Phase-Space Imaging with an Incoherent Multiscale Scattering Model

Yi Zhang,Zhi Lü,J. Wu,Xing Lin,Jianji Dong,Yeyi Cai,Jiachen Xie,Tianyi Zhu,Xiangyang Ji,Qionghai Dai

DOI: https://doi.org/10.1101/2020.11.29.402958

2020-01-01

Abstract:Optical sectioning is essential for fluorescence imaging in thick tissue to extract in-focus information from noisy background. Traditional methods achieve optical sectioning by rejecting the out-of-focus photons at a cost of photon efficiency, resulting in a tradeoff between sectioning capability and detection parallelization. Here, we show phase-space imaging with an incoherent multiscale scattering model can achieve computational optical sectioning with ~20 dB improvement for signal-to-background ratio in scattering medium, while maximizing the detection parallelization by imaging the entire volume simultaneously. We validated the superior performance by imaging various biological dynamics in Drosophila embryos, zebrafish larvae, and mice.
Shedding Light on Biology and Healthcare—preface to the Special Issue on Biomedical Optics

Peng Xi,Xunbin Wei,Junle Qu,Valery V. Tuchin

DOI: https://doi.org/10.1038/s41377-022-00804-w

2022-01-01

Abstract:This special issue collects 20 excellent papers, spanning NIR II imaging, high-speed imaging, adaptive wavefront shaping, label-free imaging, ultrasensitive detection, polarization optics, photodynamic therapy, and preclinical applications.
Roadmap on computational methods in optical imaging and holography [invited]

Joseph Rosen,Simon Alford,Blake Allan,Vijayakumar Anand,Shlomi Arnon,Francis Gracy Arockiaraj,Jonathan Art,Bijie Bai,Ganesh M Balasubramaniam,Tobias Birnbaum,Nandan S Bisht,David Blinder,Liangcai Cao,Qian Chen,Ziyang Chen,Vishesh Dubey,Karen Egiazarian,Mert Ercan,Andrew Forbes,G Gopakumar,Yunhui Gao,Sylvain Gigan,Paweł Gocłowski,Shivasubramanian Gopinath,Alon Greenbaum,Ryoichi Horisaki,Daniel Ierodiaconou,Saulius Juodkazis,Tanushree Karmakar,Vladimir Katkovnik,Svetlana N Khonina,Peter Kner,Vladislav Kravets,Ravi Kumar,Yingming Lai,Chen Li,Jiaji Li,Shaoheng Li,Yuzhu Li,Jinyang Liang,Gokul Manavalan,Aditya Chandra Mandal,Manisha Manisha,Christopher Mann,Marcin J Marzejon,Chané Moodley,Junko Morikawa,Inbarasan Muniraj,Donatas Narbutis,Soon Hock Ng,Fazilah Nothlawala,Jeonghun Oh,Aydogan Ozcan,YongKeun Park,Alexey P Porfirev,Mariana Potcoava,Shashi Prabhakar,Jixiong Pu,Mani Ratnam Rai,Mikołaj Rogalski,Meguya Ryu,Sakshi Choudhary,Gangi Reddy Salla,Peter Schelkens,Sarp Feykun Şener,Igor Shevkunov,Tomoyoshi Shimobaba,Rakesh K Singh,Ravindra P Singh,Adrian Stern,Jiasong Sun,Shun Zhou,Chao Zuo,Zack Zurawski,Tatsuki Tahara,Vipin Tiwari,Maciej Trusiak,R V Vinu,Sergey G Volotovskiy,Hasan Yılmaz,Hilton Barbosa De Aguiar,Balpreet S Ahluwalia,Azeem Ahmad

DOI: https://doi.org/10.1007/s00340-024-08280-3

Abstract:Computational methods have been established as cornerstones in optical imaging and holography in recent years. Every year, the dependence of optical imaging and holography on computational methods is increasing significantly to the extent that optical methods and components are being completely and efficiently replaced with computational methods at low cost. This roadmap reviews the current scenario in four major areas namely incoherent digital holography, quantitative phase imaging, imaging through scattering layers, and super-resolution imaging. In addition to registering the perspectives of the modern-day architects of the above research areas, the roadmap also reports some of the latest studies on the topic. Computational codes and pseudocodes are presented for computational methods in a plug-and-play fashion for readers to not only read and understand but also practice the latest algorithms with their data. We believe that this roadmap will be a valuable tool for analyzing the current trends in computational methods to predict and prepare the future of computational methods in optical imaging and holography. Supplementary information: The online version contains supplementary material available at 10.1007/s00340-024-08280-3.
Biomedical optics and photonics for advanced clinical technologies

Hossein Chamkouri,Jianmin Si,Peng Chen,Chaoshi Niu,Lei Chen

DOI: https://doi.org/10.1016/j.optlastec.2024.111265

IF: 4.939

2024-06-22

Optics & Laser Technology

Abstract:The widespread utilization of lasers and optical instruments in clinical settings to evaluate and manage patients is a testament to the profound influence that light and optical methodologies have had on contemporary medicine. Advancements in the field of biomedical optics have provided opportunities for developing more advanced technologies, particularly through integrating photonics with tissues, genetic engineering, and biomaterials. This research encompasses the basic principles of interactions between light and matter, the use of light in treatment and surgery, a concise overview of their clinical applications, and the potential of developing technologies based on light.

optics,physics, applied
Research Advances in Simple and Compact

You-Hai Liu,Tian-Xiang Qin,Ying-Ce Wang,Xing-Wang Kang,Jun Liu,Jia-Chen Wu,Liang-Cai Cao

DOI: https://doi.org/10.7498/aps.72.20230092

IF: 0.906

2023-01-01

Acta Physica Sinica

Abstract:Computational imaging enables optical imaging systems to acquire more information with miniaturized setups. Computational imaging can avoid the object-image conjugate limitation of the imaging system, and introduce encoding and decoding processes based on physical optics to achieve more efficient information transmission. It can simultaneously increase the amount of information and reduce the complexity of the system, thereby paving the way for miniaturizing imaging systems. Based on computational imaging, the simple and compact optical imaging techniques are developed, which is also called simple optics. To develop miniaturized optical imaging elements and integrated systems, simple optics utilizes the joint design of optical system and image processing algorithms, thereby realizing high-quality imaging that is comparable to complex optical systems. The imaging systems are of small-size, low-weight, and low-power consumption. With the development of micro-nano manufacturing, the optical elements have evolved from a single lens or a few lenses, to flat/planar optical elements, such as diffractive optical elements and metasurface optical elements. As a result, various lensless and metalens imaging systems have emerged. Owing to the introduction of encoding process and decoding process, an optical imaging model is developed to represent the relationship between the target object and the acquired signal, from which the computational reconstruction is used to restore the image. In the image restoration part, the algorithms are discussed in three categories, i.e. the classic algorithm, the model-based optimization iterative algorithm, and the deep learning (neural network) algorithm. Besides, the end-to-end optimization is highlighted because it introduces a new frame to minimize the complexity of optical system. In this review, the imaging techniques realized by simple optics are also discussed, such as depth imaging, high-resolution and super-resolution imaging, large field of view imaging, and extended depth of field imaging, as well as their important roles in developing consumer electronics, unmanned driving, machine vision, security monitoring, biomedical devices and metaverse. Last but not least, the challenges and future developments are prospected.
Computational metaoptics for imaging

Charles Roques-Carmes,Kai Wang,Yuanmu Yang,Arka Majumdar,Zin Lin

2024-11-14

Abstract:Metasurfaces -- ultrathin structures composed of subwavelength optical elements -- have revolutionized light manipulation by enabling precise control over electromagnetic waves' amplitude, phase, polarization, and spectral properties. Concurrently, computational imaging leverages algorithms to reconstruct images from optically processed signals, overcoming limitations of traditional imaging systems. This review explores the synergistic integration of metaoptics and computational imaging, "computational metaoptics," which combines the physical wavefront shaping ability of metasurfaces with advanced computational algorithms to enhance imaging performance beyond conventional limits. We discuss how computational metaoptics addresses the inherent limitations of single-layer metasurfaces in achieving multifunctionality without compromising efficiency. By treating metasurfaces as physical preconditioners and co-designing them with reconstruction algorithms through end-to-end (inverse) design, it is possible to jointly optimize the optical hardware and computational software. This holistic approach allows for the automatic discovery of optimal metasurface designs and reconstruction methods that significantly improve imaging capabilities. Advanced applications enabled by computational metaoptics are highlighted, including phase imaging and quantum state measurement, which benefit from the metasurfaces' ability to manipulate complex light fields and the computational algorithms' capacity to reconstruct high-dimensional information. We also examine performance evaluation challenges, emphasizing the need for new metrics that account for the combined optical and computational nature of these systems. Finally, we identify new frontiers in computational metaoptics which point toward a future where computational metaoptics may play a central role in advancing imaging science and technology.

Optics,Computer Vision and Pattern Recognition,Computational Physics,Quantum Physics

Computational Imaging: the Next Revolution for Biophotonics and Biomedicine

Integrative Approaches in Computational Biomedical Imaging

Trans-scale Optical Endoscopy Imaging Technology

Computational Photography：Keys, Methods and Applications

Optical Brain Imaging: A Powerful Tool for Neuroscience

Emerging theories and technologies on computational imaging

Recent Advances in Computational Photography

An Overview of Computational Photography

Computational Optical Coherence Tomography [Invited]

Editorial Introduction to JSTQE Special Issue on Biophotonics (2023)

High Contrast 3-D Optical Bioimaging Using Molecular and Nanoprobes Optically Responsive to IR Light

Embodied Computational Imaging: a New Paradigm for Observing and Analyzing Spatiotemporally Ultrasensitive Phenomena at Multiple Scales

IEEE Access Special Section Editorial: Biologically Inspired Image Processing Challenges and Future Directions

Deep Learning Based Computational Imaging: Status, Challenges, and Future

Research Advances in Simple and Compact Optical Imaging Techniques

Computational Optical Sectioning by Phase-Space Imaging with an Incoherent Multiscale Scattering Model

Shedding Light on Biology and Healthcare—preface to the Special Issue on Biomedical Optics

Roadmap on computational methods in optical imaging and holography [invited]

Biomedical optics and photonics for advanced clinical technologies

Research Advances in Simple and Compact

Computational metaoptics for imaging