Abstract:Existing instance segmentation techniques are primarily tailored for high-visibility inputs, but their performance significantly deteriorates in extremely low-light environments. In this work, we take a deep look at instance segmentation in the dark and introduce several techniques that substantially boost the low-light inference accuracy. The proposed method is motivated by the observation that noise in low-light images introduces high-frequency disturbances to the feature maps of neural networks, thereby significantly degrading performance. To suppress this ``feature noise", we propose a novel learning method that relies on an adaptive weighted downsampling layer, a smooth-oriented convolutional block, and disturbance suppression learning. These components effectively reduce feature noise during downsampling and convolution operations, enabling the model to learn disturbance-invariant features. Furthermore, we discover that high-bit-depth RAW images can better preserve richer scene information in low-light conditions compared to typical camera sRGB outputs, thus supporting the use of RAW-input algorithms. Our analysis indicates that high bit-depth can be critical for low-light instance segmentation. To mitigate the scarcity of annotated RAW datasets, we leverage a low-light RAW synthetic pipeline to generate realistic low-light data. In addition, to facilitate further research in this direction, we capture a real-world low-light instance segmentation dataset comprising over two thousand paired low/normal-light images with instance-level pixel-wise annotations. Remarkably, without any image preprocessing, we achieve satisfactory performance on instance segmentation in very low light (4~\% AP higher than state-of-the-art competitors), meanwhile opening new opportunities for future research.

Robust Pixel-Wise Illuminant Estimation Algorithm for Images with a Low Bit-Depth

Low-light Image Enhancement with Strong Light Weakening and Bright Halo Suppressing.

Dive into Illuminant Estimation from a Pure Color View

Efficient Illuminant Estimation for Color Constancy Using Grey Pixels

Low-Light Enhancement Using a Plug-and-Play Retinex Model With Shrinkage Mapping for Illumination Estimation

A depth iterative illumination estimation network for low-light image enhancement based on retinex theory

Robust photon-efficient imaging using a pixel-wise residual shrinkage network

Low-Light Image Enhancement via Implicit Priors Regularized Illumination Optimization

RadiantVisions: Illuminating Low-Light Imagery with a Multi-Scale Branch Network

Biologically inspired image invariance guided illuminant estimation using shallow and deep models

LLISP: Low-Light Image Signal Processing Net via Two-Stage Network

Deep Bilateral Retinex for Low-Light Image Enhancement

DI-Retinex: Digital-Imaging Retinex Theory for Low-Light Image Enhancement

ITRE: Low-light image enhancement based on illumination transmission ratio estimation

Beyond Brightening Low-light Images

Edge-guided Low-light Image Enhancement with Inertial Bregman Alternating Linearized Minimization

Self-Supervised Adaptive Illumination Estimation for Low-Light Image Enhancement

A Probabilistic Method for Image Enhancement with Simultaneous Illumination and Reflectance Estimation

An Effective Image Restorer: Denoising and Luminance Adjustment for Low-photon-count Imaging

Instance Segmentation in the Dark

Non-Uniform Illumination Image Enhancement via a Retinal Mechanism Inspired Decomposition