Robust estimation of exposure ratios in multi-exposure image stacks

Param Hanji,Rafał K. Mantiuk

DOI: https://doi.org/10.1109/TCI.2023.3301338

2023-08-12

Abstract:Merging multi-exposure image stacks into a high dynamic range (HDR) image requires knowledge of accurate exposure times. When exposure times are inaccurate, for example, when they are extracted from a camera's EXIF metadata, the reconstructed HDR images reveal banding artifacts at smooth gradients. To remedy this, we propose to estimate exposure ratios directly from the input images. We derive the exposure time estimation as an optimization problem, in which pixels are selected from pairs of exposures to minimize estimation error caused by camera noise. When pixel values are represented in the logarithmic domain, the problem can be solved efficiently using a linear solver. We demonstrate that the estimation can be easily made robust to pixel misalignment caused by camera or object motion by collecting pixels from multiple spatial tiles. The proposed automatic exposure estimation and alignment eliminates banding artifacts in popular datasets and is essential for applications that require physically accurate reconstructions, such as measuring the modulation transfer function of a display. The code for the method is available.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

The paper primarily aims to address the issue of banding artifacts that occur when merging multi-exposure image stacks into high dynamic range (HDR) images. Specifically:

1. **Banding Artifacts Issue**: When the exposure times in a multi-exposure image stack are inaccurate, banding artifacts appear in the merged HDR image. These errors often originate from the camera's EXIF metadata.

2. **Direct Estimation of Exposure Ratios**: To eliminate these banding artifacts, the authors propose a method to directly estimate the exposure ratios from the input images. By formulating the pixel values as an optimization problem in the logarithmic domain, a linear solver can be effectively used to address this issue.

3. **Noise Robustness**: Considering the impact of camera noise, the authors employ a parameterized noise model to weight different pixel correspondences and propose a robust solution under heteroscedastic noise.

4. **Spatial Balance and Multi-Exposure Fusion**: To enhance the robustness of the estimation, the authors divide the input images into multiple spatial blocks and collect pixels from each block to reduce ghosting effects caused by camera or object movement.

Through this method, the paper aims to improve the quality of existing HDR image stacks and provide better results for applications requiring physically accurate reconstructions, such as measuring the modulation transfer function of displays.