Still a Few Bugs in the System

D. Mccormick

DOI: https://doi.org/10.1038/nbt1190-981

1990-11-01

Bio/Technology

Abstract:

What problem does this paper attempt to address?

Color demosaicking via fully directional estimation

Lingyan Fan,Guorui Feng,Yanli Ren,Jinwei Wang

DOI: https://doi.org/10.1186/s40064-016-3380-1

2016-10-06

SpringerPlus

Abstract:Given a natural image from the single sensor, the key task is to properly reconstruct the full color image. This paper presents an effectively demosaicking algorithm based on fully directional estimation using Bayer color filter array pattern. The proposed method smoothly keeps access to current reconstruction implementations, and outperforms the horizontal and vertical estimating approaches in terms of the perceptual quality. To analyze the target of existing methods, the proposed algorithm use the multiscale gradients in single green channels as the diagonal information for the auxiliary interpolation. Furthermore, two group of weights (one is from the horizontal and vertical directions, another is from the diagonal and anti-diagonal directions) are built. Combinational weight is better suited for representing neighbor information. Another contribution is to better use the prior result. While calculating the same type of color difference, we divide all the color difference values into two interleaved parts. Estimated value in the first part will guide the subsequent color difference in the second part. It less brings the artifact of the interpolation procedure. Experimental results show that this adaptive algorithm is efficient both in the objective and subjective output measures.
Bayer Demosaicking With Polynomial Interpolation

Jiaji Wu,Marco Anisetti,Wei Wu,Ernesto Damiani,Gwanggil Jeon

DOI: https://doi.org/10.1109/tip.2016.2604489

IF: 10.6

2016-11-01

IEEE Transactions on Image Processing

Abstract:Demosaicking is a digital image process to reconstruct full color digital images from incomplete color samples from an image sensor. It is an unavoidable process for many devices incorporating camera sensor (e.g., mobile phones, tablet, and so on). In this paper, we introduce a new demosaicking algorithm based on polynomial interpolation-based demosaicking. Our method makes three contributions: calculation of error predictors, edge classification based on color differences, and a refinement stage using a weighted sum strategy. Our new predictors are generated on the basis of on the polynomial interpolation, and can be used as a sound alternative to other predictors obtained by bilinear or Laplacian interpolation. In this paper, we show how our predictors can be combined according to the proposed edge classifier. After populating three color channels, a refinement stage is applied to enhance the image quality and reduce demosaicking artifacts. Our experimental results show that the proposed method substantially improves over the existing demosaicking methods in terms of objective performance (CPSNR, S-CIELAB ΔE*, and FSIM), and visual performance.

computer science, artificial intelligence,engineering, electrical & electronic
An Efficient Adaptive Interpolation for Bayer CFA Demosaicking

Bin Yang,Dongsheng Wang

DOI: https://doi.org/10.1007/s11220-019-0259-6

2019-09-13

Sensing and Imaging

Abstract:Demosaicking refers to the image processing of reconstruction full color image from the raw incomplete samples recorded by a single-chip image sensor. For most available demosaicking methods, the image edges are important cues to design the interpolation scheme, and the accuracy of edge estimation has great influence on the reconstructed image quality. In this paper, a block edge estimation method is proposed by considering all the color channels comprehensively. Based on the novel edge estimation method, the proposed algorithm firstly interpolates the G channel pixels with the guidance of the color correlation and edge directions. The interpolated G channel is further used to help the R and B channel interpolations in sequence. In addition, a border pixels interpolation strategy is also presented to overcome the difficulties of the gradient estimation at border positions. Both Kodak and IMAX data set are used to test the validation of the proposed methods. The experimental results demonstrate that the proposed algorithm provides superior performances in terms of both objective evaluate and subjective visual quality.
An Improved Color Image Demosaicking Algorithm

Shousheng Luo,Haomin Zhou,Tie Zhou

DOI: https://doi.org/10.1109/cisp.2012.6469674

2012-01-01

Abstract:Color image demosaicking is an important issue in color image acquisition procedures. Most of the existing demosaicking methods reduce the artifacts at the price of the high computational cost posterior processing. In this paper, we propose a more effective demosaicking algorithm. The proposed method improves the qualities of demosaicked images by interpolation, rather than posterior processing. In order to avoid the propagation of errors occurred during green channel reconstruction, we interpolate red and blue channels by original samples to the greatest extent. Experimental results show that the proposed algorithm yields better demosaicking images as compared with a number of typical demosaicking algorithms. Furthermore, the computational cost of the proposed method is lower.
Bayer Demosaicking Using Optimised Mean Curvature over Rgb Channels

Rui Chen,Huizhu Jia,Xiange Wen,Xiaodong Xie

DOI: https://doi.org/10.1049/el.2017.1892

2017-01-01

Electronics Letters

Abstract:Color artifacts of demosaicked images are often found at contours due to interpolation across edges and cross-channel aliasing. To tackle this problem, we propose a novel demosaicking method to reliably reconstruct color channels of a Bayer image based on two different optimized mean-curvature (MC) models. The missing pixel values in green (G) channel are first estimated by minimizing a variational MC model. The curvatures of restored G-image surface are approximated as a linear MC model which guides the initial reconstruction of red (R) and blue (B) channels. Then a refinement process is performed to interpolate accurate full-resolution R and B images. Experiments on benchmark images have testified to the superiority of the proposed method in terms of both the objective and subjective quality.
Universal Demosaicking of Color Filter Arrays

Chao Zhang,Yan Li,Jue Wang,Pengwei Hao

DOI: https://doi.org/10.1109/tip.2016.2601266

IF: 10.6

2016-01-01

IEEE Transactions on Image Processing

Abstract:A large number of color filter arrays (CFAs), periodic or aperiodic, have been proposed. To reconstruct images from all different CFAs and compare their imaging quality, a universal demosaicking method is needed. This paper proposes a new universal demosaicking method based on inter-pixel chrominance capture and optimal demosaicking transformation. It skips the commonly used step to estimate the luminance component at each pixel, and thus, avoids the associated estimation error. Instead, we directly use the acquired CFA color intensity at each pixel as an input component. Two independent chrominance components are estimated at each pixel based on the inter-pixel chrominance in the window, which is captured with the difference of CFA color values between the pixel of interest and its neighbors. Two mechanisms are employed for the accurate estimation: distance-related and edge-sensing weighting to reflect the confidence levels of the inter-pixel chrominance components, and pseudoinverse-based estimation from the components in a window. Then from the acquired CFA color component and two estimated chrominance components, the three primary colors are reconstructed by a linear color transform, which is optimized for the least transform error. Our experiments show that the proposed method is much better than other published universal demosaicking methods.
An Efficient Transformer For Demosaicing Via Compressed Multi-Branch Attention Mechanism.

Xun Wu,Fanqing Meng,Yaqi Wu,Jiawei Zhang,Feng Zhang

DOI: https://doi.org/10.1109/ICASSP48485.2024.10447776

2024-01-01

Abstract:Recent demosaicing approaches are not effective and efficient enough as they do not make full use of these two factors: (1) Capturing long-range spatial dependencies effiently. (2) Reducing the computational costs when utilizing channel attention. To take them into consideration, we propose an Efficient Compressed Multibranch Transformer (ECMT) to handle demosaicing problem on various color filter arrays (CFA). There are two key specifically designed components: First, a Local-Global Spatial-wise Multi-head Self-Attention (LGSMSA) captures both spatial long-range dependencies and local context details by an efficient manner. Second, an Inner-Compressed Channel-wise Multi-head Self-Attention (ICCMSA) clusters closely related channels into the same group and then calculates channel-wise MSA within each group to reduce the computational cost. Experimental results on both real and synthetic datasets show that our ECMT achieves promising reconstruction quality in Bayer and QuadBayer CFA while requiring cheaper computational, memory costs and less inference time compared to state-of-the-art methods.
A Plug-and-play Demosaicing and Denoising Method on Mobile Cameras

Haoying Li,Honghao Chen,Meng Chang,Huajun Feng,Zhihai Xu,Qi Li,Yueting Chen

DOI: https://doi.org/10.1117/12.2606869

2021-01-01

Abstract:Image demosaicing and denoising are two important processes in the ISP pipeline of mobile cameras, because almost all mobile cameras in use today require colorful images generated by demosaicing algorithm, and the small sensor area of mobile cameras triggers low signal-to-noise ratio. Over the years, a considerable number of sequential demosaicing and denoising methods have been proposed, while they suffer from estimating the noise distribution and adjusting the hyper-parameters in order to balance demosaicing and denoising. There exit simultaneous demosaicing and denoising methods solving these problems. But they lack guidelines designed for mobile cameras. We propose a Plug-and-Play (PnP) demosaicing and denoising method on mobile cameras. Our method is built on PnP demosaicing framework which is derived from variable splitting theory. Any color demosaicing algorithm (i.e., bilinear, Malvar) can be plugged into our framework. We novelly trained an ISO conditioned denoiser for the framework and iteratively apply the denoiser in it. The ISO conditioned denoiser not only removes noise from the demosaicing procedure itself but also noise from camera sensors. By introducing ISO settings to the denoiser, our method takes possession of the adaptability and robustness in various capturing environments under different camera settings. Our method has only two hyperparameters to tune, which eases the hyper-parameter adjustment in sequential demosaicing and denoising methods. Extensive experiments on synthetic datasets show that our method performs better than sequential demosaicing and denoising methods and is practical for mobile cameras.
Multispectral Demosaicing Based on Iterative-Linear-Regression Model for Estimating Pseudo-Panchromatic Image

Kyeonghoon Jeong,Sanghoon Kim,Moon Gi Kang

DOI: https://doi.org/10.3390/s24030760

IF: 3.9

2024-01-24

Sensors

Abstract:This paper proposes a method for demosaicing raw images captured by multispectral cameras. The proposed method estimates a pseudo-panchromatic image (PPI) via an iterative-linear-regression model and utilizes the estimated PPI for multispectral demosaicing. The PPI is estimated through horizontal and vertical guided filtering, with the subsampled multispectral-filter-array-(MSFA) image and low-pass-filtered MSFA as the guide image and filtering input, respectively. The number of iterations is automatically determined according to a predetermined criterion. Spectral differences between the estimated PPI and MSFA are calculated for each channel, and each spectral difference is interpolated using directional interpolation. The weights are calculated from the estimated PPI, and each interpolated spectral difference is combined using the weighted sum. The experimental results indicate that the proposed method outperforms the State-of-the-Art methods with regard to spatial and spectral fidelity for both synthetic and real-world images.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Adaptive Multispectral Demosaicking Based on Frequency-Domain Analysis of Spectral Correlation

Sunil Prasad Jaiswal,Lu Fang,Vinit Jakhetiya,Jiahao Pang,Klaus Mueller,Oscar Chi Au

DOI: https://doi.org/10.1109/tip.2016.2634120

IF: 10.6

2017-01-01

IEEE Transactions on Image Processing

Abstract:Color filter array (CFA) interpolation, or three-band demosaicking, is a process of interpolating the missing color samples in each band to reconstruct a full color image. In this paper, we are concerned with the challenging problem of multispectral demosaicking, where each band is significantly undersampled due to the increment in the number of bands. Specifically, we demonstrate a frequency-domain analysis of the subsampled color-difference signal and observe that the conventional assumption of highly correlated spectral bands for estimating undersampled components is not precise. Instead, such a spectral correlation assumption is image dependent and rests on the aliasing interferences among the various color-difference spectra. To address this problem, we propose an adaptive spectral-correlation-based demosaicking (ASCD) algorithm that uses a novel anti-aliasing filter to suppress these interferences, and we then integrate it with an intra-prediction scheme to generate a more accurate prediction for the reconstructed image. Our ASCD is computationally very simple, and exploits the spectral correlation property much more effectively than the existing algorithms. Experimental results conducted on two data sets for multispectral demosaicking and one data set for CFA demosaicking demonstrate that the proposed ASCD outperforms the state-of-the-art algorithms.
Demosaicking Based on Channel-Correlation Adaptive Dictionary Learning

Chenyan Bai,Jia Li,Zhouchen Lin

DOI: https://doi.org/10.1117/1.jei.27.4.043047

IF: 0.829

2018-01-01

Journal of Electronic Imaging

Abstract:Image demosaicking is the problem of reconstructing color images from raw images captured using a digital camera with a color filter array. Sparse representation-based demosaicking method achieves superior performance on the commonly used Kodak dataset; however, it performs inferior on the IMAX dataset. We analyze that the factor of the sparse representation-based demosaicking methods that perform inconsistently is channel-correlation, which we define as the mean value of correlation coefficients between the RGB channels. Accordingly, we propose a channel-correlation adaptive dictionary learning-based demosaicking method. Different from the sparse representation-based demosaicking methods that use a fixed dictionary, our method trains a general dictionary on training image patches with various channel-correlations. Then, we learn a function matrix between the general dictionary and channel-correlations. For a raw image patch with an estimated channel-correlation, we compute a dictionary corresponding to its channel-correlation through the function matrix. Finally, we demosaick it with the corresponding dictionary using the sparse representation model. Experiments confirm that the proposed method performs adaptively well on raw images with various channel-correlations. (C) 2018 SPIE and IS&T
A Sparse Representation Based Joint Demosaicing Method for Single-Chip Polarized Color Sensor

Sijia Wen,Yinqiang Zheng,Feng Lu

DOI: https://doi.org/10.1109/tip.2021.3069190

IF: 10.6

2021-01-01

IEEE Transactions on Image Processing

Abstract:The emergence of the single-chip polarized color sensor now allows for simultaneously capturing chromatic and polarimetric information of the scene on a monochromatic image plane. However, unlike the usual camera with an embedded demosaicing method, the latest polarized color camera is not delivered with an in-built demosaicing tool. For demosaicing, the users have to down-sample the captured images or to use traditional interpolation techniques. Neither of them can perform well since the polarization and color are interdependent. Therefore, joint chromatic and polarimetric demosaicing is the key to obtaining high-quality polarized color images. In this paper, we propose a joint chromatic and polarimetric demosaicing model to address this challenging problem. Instead of mechanically demosaicing for the multi-channel polarized color image, we further present a sparse representation-based optimization strategy that utilizes chromatic information and polarimetric information to jointly optimize the model. To avoid the interaction between color and polarization during demosaicing, we separately construct the corresponding dictionaries. We also build an optical data acquisition system to collect a dataset, which contains various sources of polarization, such as illumination, reflectance and birefringence. Results of both qualitative and quantitative experiments have shown that our method is capable of faithfully recovering full RGB information of four polarization angles for each pixel from a single mosaic input image. Moreover, the proposed method can perform well not only on the synthetic data but the real captured data.

computer science, artificial intelligence,engineering, electrical & electronic
Low Cost Edge Sensing for High Quality Demosaicking

Yan Niu,Jihong Ouyang,Wanli Zuo,Fuxin Wang

DOI: https://doi.org/10.1109/TIP.2018.2883815

2018-06-05

Abstract:Digital cameras that use Color Filter Arrays (CFA) entail a demosaicking procedure to form full RGB images. As today's camera users generally require images to be viewed instantly, demosaicking algorithms for real applications must be fast. Moreover, the associated cost should be lower than the cost saved by using CFA. For this purpose, we revisit the classical Hamilton-Adams (HA) algorithm, which outperforms many sophisticated techniques in both speed and accuracy. Inspired by HA's strength and weakness, we design a very low cost edge sensing scheme. Briefly, it guides demosaicking by a logistic functional of the difference between directional variations. We extensively compare our algorithm with 28 demosaicking algorithms by running their open source codes on benchmark datasets. Compared to methods of similar computational cost, our method achieves substantially higher accuracy, Whereas compared to methods of similar accuracy, our method has significantly lower cost. Moreover, on test images of currently popular resolution, the quality of our algorithm is comparable to top performers, whereas its speed is tens of times faster.

Computer Vision and Pattern Recognition
Beyond Color Difference: Residual Interpolation for Color Image Demosaicking

Daisuke Kiku,Yusuke Monno,Masayuki Tanaka,Masatoshi Okutomi

DOI: https://doi.org/10.1109/tip.2016.2518082

IF: 10.6

2016-01-01

IEEE Transactions on Image Processing

Abstract:In this paper, we propose residual interpolation (RI) as an alternative to color difference interpolation, which is a widely accepted technique for color image demosaicking. Our proposed RI performs the interpolation in a residual domain, where the residuals are differences between observed and tentatively estimated pixel values. Our hypothesis for the RI is that if image interpolation is performed in a domain with a smaller Laplacian energy, its accuracy is improved. Based on the hypothesis, we estimate the tentative pixel values to minimize the Laplacian energy of the residuals. We incorporate the RI into the gradient-based threshold free algorithm, which is one of the state-of-the-art Bayer demosaicking algorithms. Experimental results demonstrate that our proposed demosaicking algorithm using the RI surpasses the state-of-the-art algorithms for the Kodak, the IMAX, and the beyond Kodak data sets.

computer science, artificial intelligence,engineering, electrical & electronic
PPI Edge Infused Spatial-Spectral Adaptive Residual Network for Multispectral Filter Array Image Demosaicing

Bowen Zhao,Jiesi Zheng,Yafei Dong,Ning Shen,Jiangxin Yang,Yanlong Cao,Yanpeng Cao

DOI: https://doi.org/10.1109/TGRS.2023.3297250

2023-01-01

Abstract:Multispectral filter array (MSFA) sensors provide a cost-effective and one-shot acquisition solution to obtain well-aligned multiband images, which are helpful for various optical and remote-sensing applications. However, the sparse spatial sampling rate and strong spectral cross correlation make MSFA image demosaicing a challenging problem. Therefore, it is essential to develop effective MSFA demosaicing solutions to reconstruct full-resolution and high-fidelity multispectral images from the raw mosaic image. In this article, we present a pseudo-panchromatic image (PPI) edge-infused spatial-spectral adaptive residual network (PPIE-SSARN) for MSFA image demosaicing. The proposed two-branch model deploys a residual subbranch to adaptively compensate for the spatial and spectral differences of reconstructed multispectral images and a PPI edge infusion subbranch to enrich the edge-related information. Moreover, we design an effective mosaic initial feature extraction module with a spatial- and spectral-adaptive weight-sharing strategy whose kernel weights can change adaptively with spatial locations and spectral bands to avoid artifacts and aliasing problems. Experimental results demonstrate the superiority of our proposed method, outperforming the state-of-the-art MSFA demosaicing approaches and achieving satisfying demosaicing results in terms of spatial accuracy and spectral fidelity. Our models and code will be publicly available.
Optimized High-Frequency Based Interpolation for Multispectral Demosaicking

Sunil Prasad Jaiswal,Lu Fang,Vinit Jakhetiya,Manohar Kuse,Oscar C. Au

DOI: https://doi.org/10.1109/icip.2016.7532880

2016-01-01

Abstract:Multispectral demosaicking, which is an extension of color demosaicking, is a challenging problem because each band is significantly undersampled and thus precise reconstruction is needed for the restoration of high-frequency components, such as edges, textures etc. In general, existing algorithms borrow high-frequency information either from different bands via inter-color correlation or from within the bands, and produces artifact in the reconstructed image. To meet this inherent shortcoming, we propose to incorporate two different high-frequency components and integrate them optimally in the linear minimum mean square sense (LMMSE) for the precise reconstruction of undersampled components. Experimental results demonstrate that the proposed algorithm based on the optimized high-frequency achieves superior performance compared to existing algorithms both in terms of objective and subjective quality.
LMMSE frequency merging for demosaicking

Weiran Tang,Oscar C. Au,Xing Wen,Yi Yang,Lu Fang

DOI: https://doi.org/10.1109/ICME.2009.5202478

2009-01-01

Abstract:For raw images captured by most digital cameras, every pixel has only on color in R, G and B. Kinds demosaicking algorithms are proposed for interpolating the missing two colors. In this article, the relationships inter and intra color channels are analyzed, and basing on the features, we propose a method to divide raw images into sub images and merge them in frequency domain with linear combination. Optimal weights are calculated with estimation values and raw values based on minimum mean square error criteria. Experiments results with different estimations are presented and discussed.
Joint learning of RGBW color filter arrays and demosaicking

Chenyan Bai,Faqi Liu,Jia Li

DOI: https://doi.org/10.1016/j.patcog.2024.110929

IF: 8

2024-08-31

Pattern Recognition

Abstract:RGBW color filter arrays (CFAs) have gained widespread attention for their superior performance in low-light conditions. Most existing demosaicking methods are tailored for specific RGBW CFAs or involve manual joint design with CFAs. The close relationship between sampling and reconstruction means that restricting the search space through predefined CFAs and demosaicking methods severely limits the ability to achieve optimal performance. In this paper, we propose a new approach for joint learning of RGBW CFA and demosaicking. This approach can simultaneously learn optimal CFAs and demosaicking methods of any size, while also being capable of reconstructing mosaicked images of any size. We use a surrogate function and arbitrary CFA interpolation to ensure end-to-end learning of the RGBW CFA. We also propose a dual-branch fusion reconstruction network that utilizes the W channel to guide the reconstruction of R, G, and B channels, reducing color errors while preserving more image details. Extensive experiments demonstrate the superiority of our proposed method. Our code is available at: https://github.com/fql0528/learncfa .

computer science, artificial intelligence,engineering, electrical & electronic
A Hardware Friendly Demosaicking Algorithm Based on Edge Sensing

Jiarui Liu,Yuxin Gao,Xiankui Xiong,Dong Xu,Xuanpeng Zhu,Yibo Fan

DOI: https://doi.org/10.1109/icsict55466.2022.9963180

2022-01-01

Abstract:Demosaicking is one of the core modules in the Image Signal Processor (ISP), which converts raw images into RGB images. The demosaicking algorithm is mainly used in ISP, which has high requirements on the complexity and accuracy of the algorithm. For this purpose, we propose a hardware friendly demosaicking algorithm based on edge sensing (EHD). We classify the edges using gradients of color difference and interpolate as far as possible along the edge direction rather than across the edge. The experimental results show that our proposed algorithm achieves high subjective quality and cPSNR of 39.15 dB, which is 0.84 dB better compared to the LED algorithm.
Model-based demosaicking for acquisitions by a RGBW color filter array

Matthieu Muller,Daniele Picone,Mauro Dalla Mura,Magnus O Ulfarsson

2023-06-02

Abstract:Microsatellites and drones are often equipped with digital cameras whose sensing system is based on color filter arrays (CFAs), which define a pattern of color filter overlaid over the focal plane. Recent commercial cameras have started implementing RGBW patterns, which include some filters with a wideband spectral response together with the more classical RGB ones. This allows for additional light energy to be captured by the relevant pixels and increases the overall SNR of the acquisition. Demosaicking defines reconstructing a multi-spectral image from the raw image and recovering the full color components for all pixels. However, this operation is often tailored for the most widespread patterns, such as the Bayer pattern. Consequently, less common patterns that are still employed in commercial cameras are often neglected. In this work, we present a generalized framework to represent the image formation model of such cameras. This model is then exploited by our proposed demosaicking algorithm to reconstruct the datacube of interest with a Bayesian approach, using a total variation regularizer as prior. Some preliminary experimental results are also presented, which apply to the reconstruction of acquisitions of various RGBW cameras.

Image and Video Processing

Still a Few Bugs in the System

Color demosaicking via fully directional estimation

Bayer Demosaicking With Polynomial Interpolation

An Efficient Adaptive Interpolation for Bayer CFA Demosaicking

An Improved Color Image Demosaicking Algorithm

Bayer Demosaicking Using Optimised Mean Curvature over Rgb Channels

Universal Demosaicking of Color Filter Arrays

An Efficient Transformer For Demosaicing Via Compressed Multi-Branch Attention Mechanism.

A Plug-and-play Demosaicing and Denoising Method on Mobile Cameras

Multispectral Demosaicing Based on Iterative-Linear-Regression Model for Estimating Pseudo-Panchromatic Image

Adaptive Multispectral Demosaicking Based on Frequency-Domain Analysis of Spectral Correlation

Demosaicking Based on Channel-Correlation Adaptive Dictionary Learning

A Sparse Representation Based Joint Demosaicing Method for Single-Chip Polarized Color Sensor

Low Cost Edge Sensing for High Quality Demosaicking

Beyond Color Difference: Residual Interpolation for Color Image Demosaicking

PPI Edge Infused Spatial-Spectral Adaptive Residual Network for Multispectral Filter Array Image Demosaicing

Optimized High-Frequency Based Interpolation for Multispectral Demosaicking

LMMSE frequency merging for demosaicking

Joint learning of RGBW color filter arrays and demosaicking

A Hardware Friendly Demosaicking Algorithm Based on Edge Sensing

Model-based demosaicking for acquisitions by a RGBW color filter array