Robust super-resolution classifier by nonlinear optics

Ishan Darji,Santosh Kumar,Yu-Ping Huang

DOI: https://doi.org/10.1364/ol.537295

IF: 3.6

2024-09-20

Optics Letters

Abstract:Ishan Darji, Santosh Kumar, Yu-Ping Huang Spatial-mode projective measurements could achieve super-resolution in remote sensing and imaging, yet their performance is usually ... [Opt. Lett. 49, 5419-5422 (2024)]

optics

What problem does this paper attempt to address?

A Robust Super-Resolution Classifier by Nonlinear Optics

Ishan Darji,Santosh Kumar,Yu-Ping Huang

DOI: https://doi.org/10.1364/OL.537295

2024-11-02

Abstract:Spatial-mode projective measurements could achieve super-resolution in remote sensing and imaging, yet their performance is usually sensitive to the parameters of the target scenes. We propose and demonstrate a robust classifier of close-by light sources by using optimized mode projection via nonlinear optics. Contrary to linear-optics based methods using the first few Hermite-Gaussian modes for the projection, here the projection modes are optimally tailored by shaping the pump wave to drive the nonlinear optical process. This minimizes modulation losses and allows high flexibility in designing those modes for robust and efficient measurements. We test this classifier on discriminating one light source and two sources separated well within the Rayleigh limit without prior knowledge of the exact centroid or brightness. Our results show a classification fidelity of over 80% even when the centroid is misaligned by half the source separation, or when one source is four times stronger than the other.

Optics
Far-field Optical Nanoscopy Via Visible Light

Xu Liu,Xiang Hao,Cuifang Kuang

DOI: https://doi.org/10.1364/n3.2013.nsu1c.2

2013-01-01

Abstract:Several newly proposed methods that can realize optical super-resolution imaging in the far-field are discussed in this paper, including their corresponding physical mechanisms, application scales, advantages, and limitations.
Robust Dual Images Super-Resolution.

Xiaohong Zhang,Yun Zhang,Guiping Qian,Aihong Qin

DOI: https://doi.org/10.1109/CISP-BMEI.2018.8633071

2018-01-01

Abstract:With the developing of 3D mobile phone, there are more and more dual images which seen from slightly different angles caught by two mobile phone cameras. The images caught by mobile phone camera usually have lower resolution (LR) than the images caught by normal camera. To solve this problem, this paper proposes a robust Bayesian SR method to recover high-frequency information from dual LR images, which solves the registration, interpolation, kernel estimation, restoration, and post-processing problems simultaneously. Itu0027s noticed that the blur kernel (Gaussian blur kernel) of the proposed SR processing is recalculated in iterations, other than a constant kernel as usual. Our experimental results show great visual improvement and a suppressing of high-frequency artifacts.
Robust and efficient single-pixel image classification with nonlinear optics

Santosh Kumar,Ting Bu,He Zhang,Irwin Huang,Yuping Huang

DOI: https://doi.org/10.1364/OL.420388

2021-04-15

Abstract:We present a hybrid image classifier by feature-sensitive image upconversion, single pixel photodetection, and deep learning, aiming at fast processing of high-resolution images. It uses partial Fourier transform to extract the images' signature features in both the original and Fourier domains, thereby significantly increasing the classification accuracy and robustness. Tested on the Modified National Institute of Standards and Technology handwritten digit images and verified by simulation, it boosts accuracy from 81.25% (by Fourier-domain processing) to 99.23%, and achieves 83% accuracy for highly contaminated images whose signal-to-noise ratio is only -17dB. Our approach could prove useful for fast lidar data processing, high-resolution image recognition, occluded target identification, and atmosphere monitoring.
Orientation-Aware Deep Neural Network for Real Image Super-Resolution.

Chen Du,He Zewei,Sun Anshun,Yang Jiangxin,Cao Yanlong,Cao Yanpeng,Tang Siliang,Michael Ying Yang

DOI: https://doi.org/10.1109/cvprw.2019.00246

2019-01-01

Abstract:Recently, Convolutional Neural Network (CNN) based approaches have achieved impressive single image super-resolution (SISR) performance in terms of accuracy and visual effects. It is noted that most SISR methods assume that the low-resolution (LR) images are obtained through bicubic interpolation down-sampling, thus their performance on real-world LR images is limited. In this paper, we proposed a novel orientation-aware deep neural network (OA-DNN) model, which incorporate a number of orientation feature extraction and channel attention modules (OAMs), to achieve good SR performance on real-world LR images captured by a single-lens reflex (DSLR) camera. Orientation-aware features extracted in different directions are adaptively combined through a channel-wise attention mechanism to generate more distinctive features for high-fidelity recovery of image details. Moreover, we reshape the input image into smaller spatial size but deeper depth via an inverse pixel-shuffle operation to accelerate the training/testing speed without sacrificing restoration accuracy. Extensive experimental results indicate that our OA-DNN model achieves a good balance between accuracy and speed. The extended OA-DNN*+ model further increases PSNR index by 0.18 dB compared with our previously submitted version. Codes will be made public after publication.
Super-resolution linear optical imaging in the far field

A.A. Pushkina,G. Maltese,J.I. Costa-Filho,P. Patel,A.I. Lvovsky

DOI: https://doi.org/10.1103/PhysRevLett.127.253602

2021-08-03

Abstract:The resolution of optical imaging devices is ultimately limited by the diffraction of light. To circumvent this limit, modern super-resolution microscopy techniques employ active interaction with the object by exploiting its optical nonlinearities, nonclassical properties of the illumination beam, or near-field probing. Thus, they are not applicable whenever such interaction is not possible, for example, in astronomy or non-invasive biological imaging. Far-field, linear-optical super-resolution techniques based on passive analysis of light coming from the object would cover these gaps. In this paper, we present the first proof-of-principle demonstration of such a technique. It works by accessing information about spatial correlations of the image optical field and, hence, about the object itself via measuring projections onto Hermite-Gaussian transverse spatial modes. With a basis of 21 spatial modes in both transverse dimensions, we perform two-dimensional imaging with twofold resolution enhancement beyond the diffraction limit.

Optics,Quantum Physics
A Super-resolution Optical Classifier with High Photon Efficiency

He Zhang,Santosh Kumar,Yu-Ping Huang

DOI: https://doi.org/10.48550/arXiv.2006.16713

2020-06-30

Quantum Physics

Abstract:We propose and demonstrate a photon-efficient optical classifier to overcome the Rayleigh limit in spatial resolution. It utilizes mode-selective sum frequency generation and single-pixel photon detection to resolve closely spaced incoherent sources based on photon counting statistics. Super-resolving and photon efficient, this technique can find applications in microscopy, light detection and ranging (LiDAR), and astrophysics.
Computational Superoscillation Imaging Beyond the Rayleigh Limit from Far-Field Measurements

Lianlin Li,Fang Li,Tie Jun Cui

DOI: https://doi.org/10.1364/oe.22.005431

IF: 3.8

2014-01-01

Optics Express

Abstract:Far-field imaging beyond the Rayleigh limit is one of the most important challenges in optics, microwave, and ultrasonics. We propose a novel sparsity-promoted super-oscillation imaging scheme for reconstructing more universal objects in subwavelength scales, which solves a weighted optimization problem constrained by lp-norm-based sparsity regularization (0≤p≤1). We demonstrate numerically that the proposed imaging technique improves the resolution related to existing approaches remarkably for the case of very high signal-to-noise ratio (SNR), including the traditional super-oscillation imaging and sparsity-based super-resolution imaging. The standard superoscillation based super-resolution imaging approach can be regarded as the first-iteration solution of the proposed scheme. Numerical results for one- and two-dimensional super-resolution imaging are presented for validation.
Adaptive Loss Function for Super Resolution Neural Networks Using Convex Optimization Techniques

Seyed Mehdi Ayyoubzadeh,Xiaolin Wu

DOI: https://doi.org/10.48550/arXiv.2001.07766

2020-01-22

Abstract:Single Image Super-Resolution (SISR) task refers to learn a mapping from low-resolution images to the corresponding high-resolution ones. This task is known to be extremely difficult since it is an ill-posed problem. Recently, Convolutional Neural Networks (CNNs) have achieved state of the art performance on SISR. However, the images produced by CNNs do not contain fine details of the images. Generative Adversarial Networks (GANs) aim to solve this issue and recover sharp details. Nevertheless, GANs are notoriously difficult to train. Besides that, they generate artifacts in the high-resolution images. In this paper, we have proposed a method in which CNNs try to align images in different spaces rather than only the pixel space. Such a space is designed using convex optimization techniques. CNNs are encouraged to learn high-frequency components of the images as well as low-frequency components. We have shown that the proposed method can recover fine details of the images and it is stable in the training process.

Computer Vision and Pattern Recognition,Machine Learning,Image and Video Processing
Robust Image/video Super-Resolution Display.

Rui Chen,Huizhu Jia,Xiaodong Xie,Wen Gao

DOI: https://doi.org/10.1145/2821592.2821629

2015-01-01

Abstract:This paper describes a new method to reconstruct high-resolution video sequences from several observed low-resolution images based on an adaptive Mumford-shah model which is extended by using nonlocal information and low-rank representation. In our regularization framework, joint image restoration and motion estimation are first implemented and then detailed information can be recovered by incorporating new model as a prior term.
Noninvasive super-resolution imaging through scattering media

Dong Wang,S. K. Sahoo,Cuong Dang

2019-06-10

Abstract:Super-resolution imaging with advanced optical systems has been revolutionizing technical analysis in various fields from biological to physical sciences. However, many objects are hidden by strongly scattering media such as rough wall corners or biological tissues that scramble light paths, create speckle patterns and hinder object's visualization, let alone super-resolution imaging. Here, we realize a method to do non-invasive super-resolution imaging through scattering media based on stochastic optical scattering localization imaging (SOSLI) technique. Simply by capturing multiple speckle patterns of photo-switchable emitters in our demonstration, the stochastic approach utilizes the speckle correlation properties of scattering media to retrieve an image with more than five-fold resolution enhancement compared to the diffraction limit, while posing no fundamental limit in achieving higher spatial resolution. More importantly, we demonstrate our SOSLI to do non-invasive super-resolution imaging through not only optical diffusers, i.e. static scattering media, but also biological tissues, i.e. dynamic scattering media with decorrelation of up to 80%. Our approach paves the way to non-invasively visualize various samples behind scattering media at unprecedented levels of detail.

Physics,Engineering
Robust super-resolution using kernel regression with outliers-reduction scheme

Xu Feng,Xu Lizhong,Huang Fengchen,Huang Chenrong,Shi Aiye

DOI: https://doi.org/10.5897/sre11.436

2011-01-01

Scientific Research and Essays

Abstract:In the process of recording a digital image, super-resolution (SR) is a feasible soft method for solving the limitation of device and effect of environment. During the last two decades, many researchers proposed various SR algorithms for image reconstruction. Among these algorithms, kernel regression is a helpful tool which considers not only spatial distance between center pixel and neighbor pixel but also structural information. However, the problem of removing noise and outlier in kernel regression can be further studied and solved. In this paper, we proposed a new idea of so called trilateral kernel regression. Besides the above factors considered, the new kernel regression with trilateral idea considers additional factor: Confident correlation of pixels, so it can obtain more accurate result. Experiments are carried out to demonstrate the effectiveness of our method. The index of RMSE can be reduced by 2 or even 4 in some severe case.
Lightweight super-resolution multimode fiber imaging with regularized linear regression

Maximilian Lipp,Wei Li,Ksenia Abrashitova,Patrick Forré,Lyubov V. Amitonova,Lyubov Amitonova

DOI: https://doi.org/10.1364/oe.522201

IF: 3.8

2024-04-10

Optics Express

Abstract:Maximilian Lipp, Wei Li, Ksenia Abrashitova, Patrick Forré, Lyubov V. Amitonova Super-resolution multimode fiber imaging provides the means to image samples quickly with compact and flexible setups finding many ... [Opt. Express 32, 15147-15155 (2024)]

optics
Robust Superresolution Computational Imaging with Diffuser-Refined Illumination

Xinliang Zhai,Xiaoyan Wu,Tailong Xiao,Jianhong Shi,Guihua Zeng

DOI: https://doi.org/10.1103/physrevapplied.20.064002

IF: 4.6

2023-01-01

Physical Review Applied

Abstract:Computational imaging allows image formation with single-pixel detectors that have no spatial resolution. It has wide-ranging potential applications in fields ranging from biomedical imaging to remote sensing. However, the practical applications of computational imaging are limited due to the high sampling ratio, weak robustness, and low resolution. In this paper, we propose a robust superresolution computational imaging scheme with diffuser-refined illumination and multibucket detection. To increase the reconstruction efficiency, a dual-stream neural network incorporated with a physical model is developed for this imaging scheme, which can reconstruct a high-resolution image in ultralow-sampling-ratio (1.56%) and ultrahigh-noise (4 dB) environments without any data for pretraining. The experimental results demonstrate that the proposed method can increase the spatial resolution by a factor of 2.59 with respect to the diffraction limit of the original imaging system. We also successfully apply this method to image a real biomedical sample and believe that this study provides a computational-imaging framework for complicated practical applications.
Super-resolution and Robust Sparse Continuous Fourier Transform in Any Constant Dimension: Nearly Linear Time and Sample Complexity

Yaonan Jin,Daogao Liu,Zhao Song

DOI: https://doi.org/10.48550/arXiv.2005.06156

2022-10-12

Abstract:The ability to resolve detail in the object that is being imaged, named by resolution, is the core parameter of an imaging system. Super-resolution is a class of techniques that can enhance the resolution of an imaging system and even transcend the diffraction limit of systems. Despite huge success in the application, super-resolution is not well understood on the theoretical side, especially for any dimension $d \geq 2$. In particular, in order to recover a $k$-sparse signal, all previous results suffer from either/both poly$(k)$ samples or running time. We design robust algorithms for any (constant) dimension under a strong noise model based on developing some new techniques in Sparse Fourier transform (Sparse FT), such as inverting a robust linear system, "eggshell" sampling schemes, and partition and voting methods in high dimension. These algorithms are the first to achieve running time and sample complexity (nearly) linear in the number of source points and logarithmic in bandwidth for any constant dimension, and we believe the techniques developed in the work can find their further applications on the Super-resolution and Sparse FT problem.

Data Structures and Algorithms
Superresolving optical ruler based on spatial mode demultiplexing for systems evolving under Brownian motion

Konrad Schlichtholz

2024-07-19

Abstract:The development of superresolution techniques, i.e., allowing for efficient resolution below the Rayleigh limit, became one of the important branches in contemporary optics and metrology. Recent findings show that perfect spatial mode demultiplexing (SPADE) into Hermite-Gauss modes followed by photon counting enables one to reach the quantum limit of precision in the task of estimation of separation between two weak stationary sources in the sub-Rayleigh regime. In order to check the limitations of the method, various imperfections such as misalignment or crosstalk between the modes were considered. Possible applications of the method in microscopy call for the adaptive measurement scheme, as the position of the measured system can evolve in time, causing non-negligible misalignment. In this paper, we examine the impact of Brownian motion of the center of the system of two weak incoherent sources of arbitrary relative brightness on adaptive SPADE measurement precision limits. The analysis is carried out using Fisher information, from which the limit of precision can be obtained by Cramér-Rao bound. As a result, we find that Rayleigh's curse is present in such a scenario; however, SPADE measurement can outperform perfect direct imaging. What is more, a suitable adjustment of the measurement time between alignments allows measurement with near-optimal precision.

Quantum Physics
Robust Multi-image Processing with Optimal Sparse Regularization

Yann Traonmilin,Saïd Ladjal,Andrés Almansa

DOI: https://doi.org/10.1007/s10851-014-0532-1

IF: 1.627

2014-08-28

Journal of Mathematical Imaging and Vision

Abstract:Sparse modeling can be used to characterize outlier type noise. Thanks to sparse recovery theory, it was shown that 1-norm super-resolution is robust to outliers if enough images are captured. Moreover, sparse modeling of signals is a way to overcome ill-posedness of under-determined problems. This naturally leads to this question: does an added sparsity assumption on the signal improve the robustness to outliers of the 1-norm super-resolution, and if yes, how strong should this assumption be? In this article, we review and extend results of the literature to the robustness to outliers of overdetermined signal recovery problems under sparse regularization, with a convex variational formulation. We then apply them to general random matrices, and show how the regularization parameter acts on the robustness to outliers. Finally, we show that in the case of multi-image processing, the structure of the support of signal and noise must be studied precisely. We show that the sparsity assumption improves robustness if outliers do not overlap with signal jumps, and determine how the regularization parameter can be chosen.

mathematics, applied,computer science, software engineering, artificial intelligence
A Robust Video Super-Resolution Algorithm.

Xinfeng Zhang,Ruiqin Xiong,Siwei Ma,Wen Gao

DOI: https://doi.org/10.1109/pcs.2010.5702567

2010-01-01

Abstract:In this paper, we propose a robust video super-resolution reconstruction method based on spatial-temporal orientation-adaptive kernel regression. First, we propose a robust registration efficiency model to reflect the temporal information reliability. Second, we propose a spatial-temporal steering kernel considering motions between frames and structures in each low resolution frame. Simulation results demonstrate that our new super-resolution method substantially improves both the subjective quality and objective quality than other resolution enhancement methods.
Confocal super-resolution microscopy based on a spatial mode sorter

Katherine K. M. Bearne,Yiyu Zhou,Boris Braverman,Jing Yang,S. A. Wadood,Andrew N. Jordan,A. N. Vamivakas,Zhimin Shi,Robert W. Boyd

DOI: https://doi.org/10.1364/OE.419493

2021-04-01

Abstract:Spatial resolution is one of the most important specifications of an imaging system. Recent results in quantum parameter estimation theory reveal that an arbitrarily small distance between two incoherent point sources can always be efficiently determined through the use of a spatial mode sorter. However, extending this procedure to a general object consisting of many incoherent point sources remains challenging, due to the intrinsic complexity of multi-parameter estimation problems. Here, we generalize the Richardson-Lucy (RL) deconvolution algorithm to address this challenge. We simulate its application to an incoherent confocal microscope, with a Zernike spatial mode sorter replacing the pinhole used in a conventional confocal microscope. We test different spatially incoherent objects of arbitrary geometry, and we find that the resolution enhancement of sorter-based microscopy is on average over 30% higher than that of a conventional confocal microscope using the standard RL deconvolution algorithm. Our method could potentially be used in diverse applications such as fluorescence microscopy and astronomical imaging.

Optics,Image and Video Processing,Quantum Physics
Robust Photometric Stereo Via Low-Rank Matrix Completion And Recovery

Lun Wu,Arvind Ganesh,Boxin Shi,Yasuyuki Matsushita,Yongtian Wang,Yi Ma

DOI: https://doi.org/10.1007/978-3-642-19318-7_55

2011-01-01

Abstract:We present a new approach to robustly solve photometric stereo problems. We cast the problem of recovering surface normals from multiple lighting conditions as a problem of recovering a low-rank matrix with both missing entries and corrupted entries, which model all types of non-Lambertian effects such as shadows and specularities. Unlike previous approaches that use Least-Squares or heuristic robust techniques, our method uses advanced convex optimization techniques that are guaranteed to find the correct low-rank matrix by simultaneously fixing its missing and erroneous entries. Extensive experimental results demonstrate that our method achieves unprecedentedly accurate estimates of surface normals in the presence of significant amount of shadows and specularities. The new technique can be used to improve virtually any photometric stereo method including uncalibrated photometric stereo.

Robust super-resolution classifier by nonlinear optics

A Robust Super-Resolution Classifier by Nonlinear Optics

Far-field Optical Nanoscopy Via Visible Light

Robust Dual Images Super-Resolution.

Robust and efficient single-pixel image classification with nonlinear optics

Orientation-Aware Deep Neural Network for Real Image Super-Resolution.

Super-resolution linear optical imaging in the far field

A Super-resolution Optical Classifier with High Photon Efficiency

Computational Superoscillation Imaging Beyond the Rayleigh Limit from Far-Field Measurements

Adaptive Loss Function for Super Resolution Neural Networks Using Convex Optimization Techniques

Robust Image/video Super-Resolution Display.

Noninvasive super-resolution imaging through scattering media

Robust super-resolution using kernel regression with outliers-reduction scheme

Lightweight super-resolution multimode fiber imaging with regularized linear regression

Robust Superresolution Computational Imaging with Diffuser-Refined Illumination

Super-resolution and Robust Sparse Continuous Fourier Transform in Any Constant Dimension: Nearly Linear Time and Sample Complexity

Superresolving optical ruler based on spatial mode demultiplexing for systems evolving under Brownian motion

Robust Multi-image Processing with Optimal Sparse Regularization

A Robust Video Super-Resolution Algorithm.

Confocal super-resolution microscopy based on a spatial mode sorter

Robust Photometric Stereo Via Low-Rank Matrix Completion And Recovery