Abstract:Recent advances of deep neural networks (DNNs) promote low-level vision applications in real-world scenarios, e.g. , image enhancement, dehazing. Nevertheless, DNN-based methods encounter challenges in terms of high computational and memory requirements, especially when deployed on real-world devices with limited resources. Quantization is one of effective compression techniques that significantly reduces computational and memory requirements by employing low-bit parameters and bit-wise operations. However, low-bit quantization for computational imaging ( Q-Imaging ) remains largely unexplored and usually suffer from a significant performance drop compared with the real-valued counterparts. In this work, through empirical analysis, we identify the main factor responsible for such significant performance drop underlies in the large gradient estimation error from non-differentiable weight quantization methods, and the activation information degeneration along with the activation quantization. To address these issues, we introduce a differentiable quantization search (DQS) method to learn the quantized weights and an information boosting module (IBM) for network activation quantization. Our DQS method allows us to treat the discrete weights in a quantized neural network as variables that can be searched. We achieve this end by using a differential approach to accurately search for these weights. In specific, each weight is represented as a probability distribution across a set of discrete values. During training, these probabilities are optimized, and the values with the highest probabilities are chosen to construct the desired quantized network. Moreover, our IBM module can rectify the activation distribution before quantization to maximize the self-information entropy, which retains the maximum information during the quantization process. Extensive experiments across a range of image processing tasks, including enhancement, super-resolution, denoising and dehazing, validate the effectiveness of our Q-Imaging along with superior performances compared to a variety of state-of-the-art quantization methods. In particular, the method in Q-Imaging also achieves a strong generalization performance when composing a detection network for the dark object detection task.

Taming Vector-Wise Quantization for Wide-Range Image Blending with Smooth Transition

DuCoFPan: Dual-Condition Flow-based Network for Pan-sharpening

Bridging the Visual Gap: Wide-Range Image Blending

Efficiently Exploiting Spatially Variant Knowledge for Video Deblurring

A Double Feature Fusion Network with Progressive Learning for Sharper Inpainting

Lightweight MIMO-WNet for Single Image Deblurring

Vector Quantized Image-to-Image Translation

Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codes

Video Vectorization via Bipartite Diffusion Curves Propagation and Optimization

Weakly Supervised Deep Hyperspherical Quantization for Image Retrieval

Hierarchical Vector-Quantized Variational Autoencoder and Vector Credibility Mechanism for High-Quality Image Inpainting

Improved Vector Quantized Diffusion Models

Decoupling Image Deblurring Into Twofold: A Hierarchical Model for Defocus Deblurring

Image Vectorization with Real-Time Thin-Plate Spline

Non-uniform motion deblurring with blurry component divided guidance

Blur-Attention: A boosting mechanism for non-uniform blurred image restoration

Edge-guided Composition Network for Image Stitching

An image deblurring method using improved U-Net model based on multilayer fusion and attention mechanism

VOGTNet: Variational Optimization-Guided Two-Stage Network for Multispectral and Panchromatic Image Fusion

Taming Scalable Visual Tokenizer for Autoregressive Image Generation

Learning Accurate Low-bit Quantization towards Efficient Computational Imaging