Abstract:Some recent methods for lossy signal and image compression store only a few selected pixels and fill in the missing structures by inpainting with a partial differential equation (PDE). Suitable operators include the Laplacian, the biharmonic operator, and edge-enhancing anisotropic diffusion (EED). The quality of such approaches depends substantially on the selection of the data that is kept. Optimising this data in the domain and codomain gives rise to challenging mathematical problems that shall be addressed in our work. In the 1D case, we prove results that provide insights into the difficulty of this problem, and we give evidence that a splitting into spatial and tonal (i.e. function value) optimisation does hardly deteriorate the results. In the 2D setting, we present generic algorithms that achieve a high reconstruction quality even if the specified data is very sparse. To optimise the spatial data, we use a probabilistic sparsification, followed by a nonlocal pixel exchange that avoids getting trapped in bad local optima. After this spatial optimisation we perform a tonal optimisation that modifies the function values in order to reduce the global reconstruction error. For homogeneous diffusion inpainting, this comes down to a least squares problem for which we prove that it has a unique solution. We demonstrate that it can be found efficiently with a gradient descent approach that is accelerated with fast explicit diffusion (FED) cycles. Our framework allows to specify the desired density of the inpainting mask a priori. Moreover, is more generic than other data optimisation approaches for the sparse inpainting problem, since it can also be extended to nonlinear inpainting operators such as EED. This is exploited to achieve reconstructions with state-of-the-art quality. We also give an extensive literature survey on PDE-based image compression methods.

Geometry of the Visual Cortex with Applications to Image Inpainting and Enhancement

Made in the humanities: Dual integral invariants for efficient edge detection

Curvature Integration in a 5D Kernel for Extracting Vessel Connections in Retinal Images

Contourlet transform-based sharpening enhancement of retinal images and vessel extraction application

Adaptive Feature Enhancement of Retinal Vascular Images Using Wavelet-Based Multiresolution Analysis

Optimising Spatial and Tonal Data for PDE-based Inpainting

A nonlocal feature-driven exemplar-based approach for image inpainting

An efficient framework for image/video inpainting

Optimal Transport on the Lie Group of Roto-translations

A note on the convergence of multigrid methods for the Riesz-space equation and an application to image deblurring

Contour-guided Image Completion with Perceptual Grouping

Compensated Convexity Methods for Approximations and Interpolations of Sampled Functions in Euclidean Spaces: Applications to Contour Lines, Sparse Data and Inpainting

Problems and related results in Algebraic Vision and Multiview Geometry

Efficient Parallel Data Optimization for Homogeneous Diffusion Inpainting of 4K Images

Cortical Functional architectures as contact and sub-Riemannian geometry

Mathematical Models for Local Nontexture Inpaintings

A Linear Framework for Region-Based Image Segmentation and Inpainting Involving Curvature Penalization

A new perspective on border completion in visual cortex as bicycle rear wheel geodesics paths via sub-Riemannian Hamiltonian formalism

Stippling and Silhouettes Rendering in Geometry-Image Space.

Dual-Tree Complex Wavelet Transform and Deep CNN-Based Super-Resolution for Video Inpainting with Application to Object Removal and Error Concealment

Understanding physiological and degenerative natural vision mechanisms to define contrast and contour operators