Approach To Solve Ill-Posed Inverse Problem In Image Reconstruction From Projections

Weihui Dai

2009-01-01

Abstract:This paper discusses the ill-posed inverse problem in image reconstruction from projections and presents a new approach, Multi-criterion Optimization Approach, to solve this problem. Theoretical analysis and test result show that the new approach provides high stability and accuracy with good recovery of visual features.

What problem does this paper attempt to address?

Multi-Objective Optimization Approach to Ill-Posed Inverse Problem

Dai Weihui,Zhang Liu,Liang Shuyi

DOI: https://doi.org/10.1109/icnc.2008.863

2008-01-01

Abstract:Influenced by ill-posedness, the solution of inverse problems is usually unstable, not unique or does not exit. One of the characteristics which image reconstruction differs from other inverse problems is that the former emphasizes not only the solution accuracy, but also its fidelity based on various visual features. Multi-objective optimization approach, a new approach to solve the ill-posed inverse problems in image reconstruction, was presented with its theoretical basis and the experiment results. Relationship between the accuracy and weighted coefficients of each single objective function was discussed with a set of test data.
Multi-criterion Optimization Approach to Ill-posed Inverse Problem with Visual Feature’s Recovery

Weihui Dai

DOI: https://doi.org/10.4304/jmm.5.3.240-247

2010-01-01

Journal of Multimedia

Abstract:Ill-posed inverse problem is commonly existed in signal processing such as image reconstruction from projection, parameter estimation on e lectromagnetic field, and path optimization in IP network. Usually, the solution of an inverse problem is unstable, not uniqu e or does not exi t. Traditional approach to solve this problem is to estimate the solution by optimizing a regularized objective function. In some cases, recovery of visual features is most emphasized in that solution; thereof the distribution of residual errors has distinct influence on the quality of solution. This paper analyzes ill-posed inverse problem with the case of image reconstruction from projections and discusses its fidelity based on various visual features in the estimated solution. Multi -criterion Optimization Approac h , a new approach to solve the ill-posed inverse problems with good recovery of visual features, is presented with its theory basis and the experiment results. The solution stability and accuracy are analyzed using singular value decomposition (SVD), and main factors affecting the reconstruction quality are also discussed.
Researches on Multicriterion Optimization Approach to Image Reconstruction

Dai Weihui,Liang Shuyi,Zhou Xuan

DOI: https://doi.org/10.1109/isip.2008.146

2008-01-01

Abstract:In most cases, ill-posedness exists inherently in the solution process of inverse image problems (such as X-ray CT image reconstruction from projections, ECG and EEG inverse mapping), and affects seriously the quality, stability and accuracy of the reconstructed images with the susceptibility to the errors in measurement data and numerical solution of forward problems. This paper gave a study on the ill-posedness of image reconstruction from projections and discuss the characteristics of the ghost function which affects imaging quality and accuracy in the reconstruction process. Multicriterion regularization approach, a new approach to solve the ill-posed inverse problems, was presented with its theory basis and the experiment results. The solution stability and accuracy were studied using singular value decomposition (SVD) theory, and main factors affecting the reconstruction quality are discussed.
Random mesh projectors for inverse problems

Sidharth Gupta,Konik Kothari,Maarten V. de Hoop,Ivan Dokmanić

DOI: https://doi.org/10.48550/arXiv.1805.11718

2018-12-06

Abstract:We propose a new learning-based approach to solve ill-posed inverse problems in imaging. We address the case where ground truth training samples are rare and the problem is severely ill-posed - both because of the underlying physics and because we can only get few measurements. This setting is common in geophysical imaging and remote sensing. We show that in this case the common approach to directly learn the mapping from the measured data to the reconstruction becomes unstable. Instead, we propose to first learn an ensemble of simpler mappings from the data to projections of the unknown image into random piecewise-constant subspaces. We then combine the projections to form a final reconstruction by solving a deconvolution-like problem. We show experimentally that the proposed method is more robust to measurement noise and corruptions not seen during training than a directly learned inverse.

Computer Vision and Pattern Recognition,Machine Learning,Image and Video Processing
Image Reconstruction without Explicit Priors

Angela F. Gao,Oscar Leong,He Sun,Katherine L. Bouman

2023-03-22

Abstract:We consider solving ill-posed imaging inverse problems without access to an explicit image prior or ground-truth examples. An overarching challenge in inverse problems is that there are many undesired images that fit to the observed measurements, thus requiring image priors to constrain the space of possible solutions to more plausible reconstructions. However, in many applications it is difficult or potentially impossible to obtain ground-truth images to learn an image prior. Thus, inaccurate priors are often used, which inevitably result in biased solutions. Rather than solving an inverse problem using priors that encode the explicit structure of any one image, we propose to solve a set of inverse problems jointly by incorporating prior constraints on the collective structure of the underlying images.The key assumption of our work is that the ground-truth images we aim to reconstruct share common, low-dimensional structure. We show that such a set of inverse problems can be solved simultaneously by learning a shared image generator with a low-dimensional latent space. The parameters of the generator and latent embedding are learned by maximizing a proxy for the Evidence Lower Bound (ELBO). Once learned, the generator and latent embeddings can be combined to provide reconstructions for each inverse problem. The framework we propose can handle general forward model corruptions, and we show that measurements derived from only a few ground-truth images (O(10)) are sufficient for image reconstruction without explicit priors.

Image and Video Processing,Computer Vision and Pattern Recognition
3d Reconstruction With Multi-View Texture Mapping

Xiaodan Ye,Lianghao Wang,Dongxiao Li,Ming Zhang

DOI: https://doi.org/10.1007/978-3-319-70090-8_21

2017-01-01

Abstract:In this paper, a novel 3D reconstruction with multi-view texture mapping method based on Kinect 2 is proposed. Camera poses of all chosen key frames are optimized according to photometric consistency. Optimized camera poses can make the projected point from vertices to different views get closer. A small range of translations with limited calculation is added in this method. A new form of data term and smoothness term in Markov Random Field (MRF) objective function is presented. The outlier images are rejected before view selection and Poisson blending are applied in the end. Experimental results show that our method achieves a high-quality 3D model with high fidelity texture.
Multiview Textured Mesh Recovery by Differentiable Rendering

Lixiang Lin,Jianke Zhu,Yisu Zhang

DOI: https://doi.org/10.1109/tcsvt.2022.3213543

IF: 5.859

2023-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Although having achieved the promising results on shape and color recovery through self-supervision, the multi-layer perceptrons-based methods usually suffer from heavy computational cost on learning the deep implicit surface representation. Since rendering each pixel requires a forward network inference, it is very computationally intensive to synthesize a whole image. To tackle these challenges, we propose an effective coarse-to-fine approach to recover the textured mesh from multi-views in this paper. Specifically, a differentiable Poisson Solver is employed to represent the object’s shape, which is able to produce topology-agnostic and watertight surfaces. To account for depth information, we optimize the shape geometry by minimizing the differences between the rendered mesh and the predicted depth from multi-view stereo. In contrast to the implicit neural representation on shape and color, we introduce a physically-based inverse rendering scheme to jointly estimate the environment lighting and object’s reflectance, which is able to render the high resolution image at real-time. The texture of reconstructed mesh is interpolated from a learnable dense texture grid. We have conducted the extensive experiments on several multi-view stereo datasets, whose promising results demonstrate the efficacy of our proposed approach. The code is available at https://github.com/l1346792580123/diff .
Optimization-based Reconstruction of Sparse Images from Few-View Projections.

Xiao Han,Junguo Bian,Erik L. Ritman,Emil Y. Sidky,Xiaochuan Pan

DOI: https://doi.org/10.1088/0031-9155/57/16/5245

IF: 3.5

2012-01-01

Physics in Medicine and Biology

Abstract:In this work, we investigate optimization-based image reconstruction from few-view (i.e. less than ten views) projections of sparse objects such as coronary-artery specimens. Using optimization programs as a guide, we formulate constraint programs as reconstruction programs and develop algorithms to reconstruct images through solving the reconstruction programs. Characterization studies are carried out for elucidating the algorithm properties of 'convergence' (relative to designed solutions) and 'utility' (relative to desired solutions) by using simulated few-view data calculated from a discrete FORBILD coronary-artery phantom, and real few-view data acquired from a human coronary-artery specimen. Study results suggest that carefully designed reconstruction programs and algorithms can yield accurate reconstructions of sparse images from few-view projections.
Iterative reconstruction for limited-angle CT using implicit neural representation

Jooho Lee,Jongduk Baek

DOI: https://doi.org/10.1088/1361-6560/ad3c8e

IF: 3.5

2024-04-11

Physics in Medicine and Biology

Abstract:Objective. Limited-angle computed tomography (CT) presents a challenge due to its ill-posed nature. In such scenarios, analytical reconstruction methods often exhibit severe artifacts. To tackle this inverse problem, several supervised deep learning-based approaches have been proposed. However, they are constrained by limitations such as generalization issue and the difficulty of acquiring a large amount of paired CT images. Approach. In this work, we propose an iterative neural reconstruction framework designed for limited-angle CT. By leveraging a coordinate-based neural representation, we formulate tomographic reconstruction as a convex optimization problem involving a deep neural network. We then employ differentiable projection layer to optimize this network by minimizing the discrepancy between the predicted and measured projection data. In addition, we introduce a prior-based weight initialization method to ensure the network starts optimization with an informed initial guess. This strategic initialization significantly improves the quality of iterative reconstruction by stabilizing the divergent behavior in ill-posed neural fields. Our method operates in a self-supervised manner, thereby eliminating the need for extensive data. Main results. The proposed method outperforms other iterative and learning-based methods. Experimental results on XCAT and Mayo Clinic datasets demonstrate the effectiveness of our approach in restoring anatomical features as well as structures. This finding was substantiated by visual inspections and quantitative evaluations using NRMSE, PSNR, and SSIM. Moreover, we conduct a comprehensive investigation into the divergent behavior of iterative neural reconstruction, thus revealing its suboptimal convergence when starting from scratch. In contrast, our method consistently produced accurate images by incorporating an initial estimate as informed initialization. Significance. This work showcases the feasibility to reconstruct high-fidelity CT images from limited-angle X-ray projections. The proposed methodology introduces a novel data-free approach to enhance medical imaging, holding promise across various clinical applications.

engineering, biomedical,radiology, nuclear medicine & medical imaging
Hybrid Projection Methods with Recycling for Inverse Problems

Jiahua Jiang,Julianne Chung,Eric de Sturler

DOI: https://doi.org/10.1137/20M1349515

2021-05-05

SIAM Journal of Scientific Computing

Abstract:SIAM Journal on Scientific Computing, Ahead of Print. Iterative hybrid projection methods have proven to be very effective for solving large linear inverse problems due to their inherent regularizing properties as well as the added flexibility to select regularization parameters adaptively. In this work, we develop Golub--Kahan-based hybrid projection methods that can exploit compression and recycling techniques in order to solve a broad class of inverse problems where memory requirements or high computational cost may otherwise be prohibitive. For problems that have many unknown parameters and require many iterations, hybrid projection methods with recycling can be used to compress and recycle the solution basis vectors to reduce the number of solution basis vectors that must be stored, while obtaining a solution accuracy that is comparable to that of standard methods. If reorthogonalization is required, this may also reduce computational cost substantially. In other scenarios, such as streaming data problems or inverse problems with multiple datasets, hybrid projection methods with recycling can be used to efficiently integrate previously computed information for faster and better reconstruction. Additional benefits of the proposed methods are that various subspace selection and compression techniques can be incorporated, standard techniques for automatic regularization parameter selection can be used, and the methods can be applied multiple times in an iterative fashion. Theoretical results show that, under reasonable conditions, regularized solutions for our proposed recycling hybrid method remain close to regularized solutions for standard hybrid methods and reveal important connections among the resulting projection matrices. Numerical examples from image processing show the potential benefits of combining recycling with hybrid projection methods.
Deep Convolutional Neural Network for Inverse Problems in Imaging

Kyong Hwan Jin,Michael T. McCann,Emmanuel Froustey,Michael Unser

DOI: https://doi.org/10.1109/TIP.2017.2713099

IF: 10.6

2022-03-15

IEEE Transactions on Image Processing

Abstract:In this paper, we propose a novel deep convolutional neural network (CNN)-based algorithm for solving ill-posed inverse problems. Regularized iterative algorithms have emerged as the standard approach to ill-posed inverse problems in the past few decades. These methods produce excellent results, but can be challenging to deploy in practice due to factors including the high computational cost of the forward and adjoint operators and the difficulty of hyperparameter selection. The starting point of this paper is the observation that unrolled iterative methods have the form of a CNN (filtering followed by pointwise nonlinearity) when the normal operator (H*H, where H* is the adjoint of the forward imaging operator, H) of the forward model is a convolution. Based on this observation, we propose using direct inversion followed by a CNN to solve normal-convolutional inverse problems. The direct inversion encapsulates the physical model of the system, but leads to artifacts when the problem is ill posed; the CNN combines multiresolution decomposition and residual learning in order to learn to remove these artifacts while preserving image structure. We demonstrate the performance of the proposed network in sparse-view reconstruction (down to 50 views) on parallel beam X-ray computed tomography in synthetic phantoms as well as in real experimental sinograms. The proposed network outperforms total variation-regularized iterative reconstruction for the more realistic phantoms and requires less than a second to reconstruct a 512 × 512 image on the GPU.

computer science, artificial intelligence,engineering, electrical & electronic
Improving the Reconstruction Accuracy of Mr Imaging Using Zero-Point Attracting Projection

Kai Qiu,Hua Guo,Yuantao Gu

DOI: https://doi.org/10.1109/chicc.2016.7554165

2016-01-01

Abstract:In order to reduce the scanning time and improve the reconstruction accuracy, the application of compressive sensing in MR image reconstruction has been widely studied recently. Most of the literatures formulate the reconstruction as a convex ℓ 1 -minimization problem and show considerable improvement. However, some other researches show that non-convex penalties may induce better sparsity than convex ones, and achieve better reconstruction accuracy. In this paper, an efficient algorithm based on Zero-point Attracting Projection is proposed. Specifically, the concept of weak convexity is used to characterize the sparsity of image, and the reconstruction is formulated as a non-convex optimization problem, which is solved by generalized gradient projection. Experimental results demonstrate that the proposed algorithm exhibits better performance in terms of signal-to-noise ratio and artifact power compared with previous methods.
Iterative algorithms applied in research of tomography image reconstruction with incomplete projection data

冷帅,张丽,陈志强,赵自然

DOI: https://doi.org/10.3969/j.issn.0258-0934.2003.03.007

2003-01-01

Abstract:Projection data gained during Industrial Computer Tomography (ICT) measurement is incomplete sometimes for particular reasons. Then the image quality reconstructed by conventional analytical algorithms will be unsatisfactory. The implementation of iterative algorithms can improve this condition. Gradient Algorithm and Convex Algorithm based on MaximumLikelihood (ML) are applied in reconstruction with incomplete projection data.
Polarimetric Multi-View Inverse Rendering

Jinyu Zhao,Yusuke Monno,Masatoshi Okutomi

DOI: https://doi.org/10.1109/TPAMI.2022.3232211

Abstract:A polarization camera has great potential for 3D reconstruction since the angle of polarization (AoP) and the degree of polarization (DoP) of reflected light are related to an object's surface normal. In this paper, we propose a novel 3D reconstruction method called Polarimetric Multi-View Inverse Rendering (Polarimetric MVIR) that effectively exploits geometric, photometric, and polarimetric cues extracted from input multi-view color-polarization images. We first estimate camera poses and an initial 3D model by geometric reconstruction with a standard structure-from-motion and multi-view stereo pipeline. We then refine the initial model by optimizing photometric rendering errors and polarimetric errors using multi-view RGB, AoP, and DoP images, where we propose a novel polarimetric cost function that enables an effective constraint on the estimated surface normal of each vertex, while considering four possible ambiguous azimuth angles revealed from the AoP measurement. The weight for the polarimetric cost is effectively determined based on the DoP measurement, which is regarded as the reliability of polarimetric information. Experimental results using both synthetic and real data demonstrate that our Polarimetric MVIR can reconstruct a detailed 3D shape without assuming a specific surface material and lighting condition.
Theoretical Approach of Single Lens Based 3D Imaging Reconstruction and Its Computational Analysis

Guanghua Gong,Hongming Zhang,Minyu Yao

DOI: https://doi.org/10.1364/math.2017.mtu3c.4

2017-01-01

Abstract:We propose a theoretical approach of single lens based 3D imaging reconstruction and its computational analysis. It aims to establish the mapping between the object space and image space by means of point spread function.
Mathematical problems of joint reconstruction for biomedical multi-modality imaging techniques

Ming JIANG,Di HE

DOI: https://doi.org/10.1360/N012015-00395

2016-01-01

Abstract:Biomedical imaging aims at visualizing structural or functional information necessary for biological research or clinical diagnosis. Mathematics has played a significant role in biomedical imaging technology. A class of mathematical problems in biomedical imaging techniques is to reconstruct the information of interest from measured data, which typically poses an ill-posed inverse problem. One recent development is the multi-modality imaging technique for enriching image information by fusing multiple imaging modalities. In contrast to the existing single-mode imaging technology, there is a variety of strategies for image reconstruction of multi-modality imaging techniques. A new direction is the use of the similarity of multi-modality images, or more exactly, the use of the partial similarity of image features from different modality images for achieving the joint reconstruction of multi-modality images. The advantages of joint reconstruction are to utilize the data from different modalities to steer the reconstruction of one modality through the complementary feature information from another modality and to get better quality images. This note is a review of mathematical problems of the joint reconstruction for multi-modality imaging techniques. We introduce the key mathematical problems and report relevant progresses in this field. The fundamental Mumford-Shah functional in this approach has been discoursed in a book by Zhang Gongqing.
An Extended Primal-Dual Algorithm Framework for Nonconvex Problems with Application to Nonlinear Imaging

Yu Gao,Xiaochuan Pan,Chong Chen

DOI: https://doi.org/10.1088/1361-6420/ac79c8

2021-09-15

Abstract:We propose an extended primal-dual algorithm framework for solving a general nonconvex optimization model. This work is motivated by image reconstruction problems in a class of nonlinear imaging, where the forward operator can be formulated as a nonlinear convex function with respect to the reconstructed image. Using the proposed framework, we put forward six specific iterative schemes, and present their detailed mathematical explanation. We also establish the relationship to existing algorithms. Moreover, under proper assumptions, we analyze the convergence of the schemes for the general model when the optimal dual variable regarding the nonlinear operator is non-vanishing. As a representative, the image reconstruction for spectral computed tomography is used to demonstrate the effectiveness of the proposed algorithm framework. By special properties of the concrete problem, we further prove the convergence of these customized schemes when the optimal dual variable regarding the nonlinear operator is vanishing. Finally, the numerical experiments show that the proposed algorithm has good performance on image reconstruction for various data with non-standard scanning configuration.

Optimization and Control,Medical Physics
Image Reconstruction via Deep Image Prior Subspaces

Riccardo Barbano,Javier Antorán,Johannes Leuschner,José Miguel Hernández-Lobato,Bangti Jin,Željko Kereta

2023-06-05

Abstract:Deep learning has been widely used for solving image reconstruction tasks but its deployability has been held back due to the shortage of high-quality training data. Unsupervised learning methods, such as the deep image prior (DIP), naturally fill this gap, but bring a host of new issues: the susceptibility to overfitting due to a lack of robust early stopping strategies and unstable convergence. We present a novel approach to tackle these issues by restricting DIP optimisation to a sparse linear subspace of its parameters, employing a synergy of dimensionality reduction techniques and second order optimisation methods. The low-dimensionality of the subspace reduces DIP's tendency to fit noise and allows the use of stable second order optimisation methods, e.g., natural gradient descent or L-BFGS. Experiments across both image restoration and tomographic tasks of different geometry and ill-posedness show that second order optimisation within a low-dimensional subspace is favourable in terms of optimisation stability to reconstruction fidelity trade-off.

Computer Vision and Pattern Recognition,Image and Video Processing
Unsupervised approaches based on optimal transport and convex analysis for inverse problems in imaging

Marcello Carioni,Subhadip Mukherjee,Hong Ye Tan,Junqi Tang

DOI: https://doi.org/10.48550/arXiv.2311.08972

2023-11-29

Abstract:Unsupervised deep learning approaches have recently become one of the crucial research areas in imaging owing to their ability to learn expressive and powerful reconstruction operators even when paired high-quality training data is scarcely available. In this chapter, we review theoretically principled unsupervised learning schemes for solving imaging inverse problems, with a particular focus on methods rooted in optimal transport and convex analysis. We begin by reviewing the optimal transport-based unsupervised approaches such as the cycle-consistency-based models and learned adversarial regularization methods, which have clear probabilistic interpretations. Subsequently, we give an overview of a recent line of works on provably convergent learned optimization algorithms applied to accelerate the solution of imaging inverse problems, alongside their dedicated unsupervised training schemes. We also survey a number of provably convergent plug-and-play algorithms (based on gradient-step deep denoisers), which are among the most important and widely applied unsupervised approaches for imaging problems. At the end of this survey, we provide an overview of a few related unsupervised learning frameworks that complement our focused schemes. Together with a detailed survey, we provide an overview of the key mathematical results that underlie the methods reviewed in the chapter to keep our discussion self-contained.

Computer Vision and Pattern Recognition,Machine Learning,Optimization and Control
3D Motion Reconstruction from Perspective Projection

陈家实,庄越挺,朱强,潘云鹤

DOI: https://doi.org/10.3321/j.issn:1003-9775.2002.11.009

2002-01-01

Abstract:Due to lack of information in 3-dimension, the problem of generating 3D human motion from 2D images taken by single camera is almost impossible at first sight and is regarded as an ill-conditioned problem. Our idea is based on a propagation model and an algorithm is proposed to acquire the 3D motion information. In contrast to the existing methods, our approach utilizes a set of rules to get the smooth starting points of the model within a predefined space. Then the spatial ambiguity arisen from the fact that there are two possible poses of a link in 3D is eliminated by our expanding algorithm. Finally, several experiment results are presented and validated.

Approach To Solve Ill-Posed Inverse Problem In Image Reconstruction From Projections

Multi-Objective Optimization Approach to Ill-Posed Inverse Problem

Multi-criterion Optimization Approach to Ill-posed Inverse Problem with Visual Feature’s Recovery

Researches on Multicriterion Optimization Approach to Image Reconstruction

Random mesh projectors for inverse problems

Image Reconstruction without Explicit Priors

3d Reconstruction With Multi-View Texture Mapping

Multiview Textured Mesh Recovery by Differentiable Rendering

Optimization-based Reconstruction of Sparse Images from Few-View Projections.

Iterative reconstruction for limited-angle CT using implicit neural representation

Hybrid Projection Methods with Recycling for Inverse Problems

Deep Convolutional Neural Network for Inverse Problems in Imaging

Improving the Reconstruction Accuracy of Mr Imaging Using Zero-Point Attracting Projection

Iterative algorithms applied in research of tomography image reconstruction with incomplete projection data

Polarimetric Multi-View Inverse Rendering

Theoretical Approach of Single Lens Based 3D Imaging Reconstruction and Its Computational Analysis

Mathematical problems of joint reconstruction for biomedical multi-modality imaging techniques

An Extended Primal-Dual Algorithm Framework for Nonconvex Problems with Application to Nonlinear Imaging

Image Reconstruction via Deep Image Prior Subspaces

Unsupervised approaches based on optimal transport and convex analysis for inverse problems in imaging

3D Motion Reconstruction from Perspective Projection