Abstract:The quality of images captured by smartphones is an important specification since smartphones are becoming ubiquitous as primary capturing devices. The traditional image signal processing (ISP) pipeline in a smartphone camera consists of several image processing steps performed sequentially to reconstruct a high quality sRGB image from the raw sensor data. These steps consist of demosaicing, denoising, white balancing, gamma correction, colour enhancement, etc. Since each of them are performed sequentially using hand-crafted algorithms, the residual error from each processing module accumulates in the final reconstructed signal. Thus, the traditional ISP pipeline has limited reconstruction quality in terms of generalizability across different lighting conditions and associated noise levels while capturing the image. Deep learning methods using convolutional neural networks (CNN) have become popular in solving many image-related tasks such as image denoising, contrast enhancement, super resolution, deblurring, etc. Furthermore, recent approaches for the RAW to sRGB conversion using deep learning methods have also been published, however, their immense complexity in terms of their memory requirement and number of Mult-Adds make them unsuitable for mobile camera ISP. In this paper we propose DelNet - a single end-to-end deep learning model - to learn the entire ISP pipeline within reasonable complexity for smartphone deployment. Del-Net is a multi-scale architecture that uses spatial and channel attention to capture global features like colour, as well as a series of lightweight modified residual attention blocks to help with denoising. For validation, we provide results to show the proposed Del-Net achieves compelling reconstruction quality.

PyNET-QxQ: An Efficient PyNET Variant for QxQ Bayer Pattern Demosaicing in CMOS Image Sensors

Extreme-Quality Computational Imaging via Degradation Framework

A Plug-and-play Demosaicing and Denoising Method on Mobile Cameras

QCGFNet: A Lightweight Color Image Denoising Network

PyNET-CA: Enhanced PyNET with Channel Attention for End-to-End Mobile Image Signal Processing

Efficient Unified Demosaicing for Bayer and Non-Bayer Patterned Image Sensors

Learning to Joint Remosaic and Denoise in Quad Bayer CFA via Universal Multi-scale Channel Attention Network.

Inheriting Bayer's Legacy-Joint Remosaicing and Denoising for Quad Bayer Image Sensor

Inheriting Bayer's Legacy: Joint Remosaicing and Denoising for Quad Bayer Image Sensor

Del-Net: A Single-Stage Network for Mobile Camera ISP

Modeling Dual-Exposure Quad-Bayer Patterns for Joint Denoising and Deblurring

DemosaicFormer: Coarse-to-Fine Demosaicing Network for HybridEVS Camera

Joint Demosaicing and Denoising with Gradient Guidance in Quad Bayer CFA

Learning Rich Information for Quad Bayer Remosaicing and Denoising.

SYENet: A Simple Yet Effective Network for Multiple Low-Level Vision Tasks with Real-time Performance on Mobile Device

PyNet-V2 Mobile: Efficient On-Device Photo Processing With Neural Networks

Beyond Joint Demosaicking and Denoising: An Image Processing Pipeline for a Pixel-bin Image Sensor

PQDynamicISP: Dynamically Controlled Image Signal Processor for Any Image Sensors Pursuing Perceptual Quality

Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 Challenge: Report

Disparity Estimation Using a Quad-Pixel Sensor

Dynamic Low-light Imaging with Quanta Image Sensors