Gradientshop: A perceptually-motivated optimization-framework for image and video processing

Pravin Bhat, C Lawrence Zitnick, Michael Cohen, Brian Curless
2008-01-01
Abstract:We present an optimization framework for expressing image processing applications that can account for three perceptual biases of the human visual system (HVS) already well-known in the perception literature.(1) The perception literature is ripe with studies demonstrating the HVS to be more sensitive to local pixel gradients than absolute pixel values, which has led to some important work in gradient-domain image-filtering. Inspired by this work, our optimization framework allows image and video processing applications to easily specify both zeroth order constraints (ie, desired pixel values) and first order constraints (ie, desired pixel gradients in space and time) in the optimization.(2) We introduce a spatiallyvarying weighting scheme for these constraints that reduces artifacts by approximating the more robust L1-norm even when using a simple weighted least squares optimization.(3) We also demonstrate that edge length in addition to local gradient magnitude is a useful measure of local gradient saliency. Our saliency measure is inspired by perception studies that show long coherent edges in an image, even when faint, are perceptually salient to the HVS.Finally, we demonstrate the utility of our formulation in creating effective yet simple to implement solutions for common image processing tasks. To exercise our formulation we have created a new saliency-based sharpen filter and a pseudo image relighting application. We also revisit and improve upon filters previously defined by the gradient domain community–filters like painterly rendering, image de-blocking, and sparse data interpolation over images (eg, colorization using optimization).
What problem does this paper attempt to address?