Abstract:This paper begins with a description of methods for estimating image probability density functions that reflects the observation that such data is usually constrained to lie in restricted regions of the high-dimensional image space-not every pattern of pixels is an image. It is common to say that images lie on a lower-dimensional manifold in the high-dimensional space. However, it is not the case that all points on the manifold have an equal probability of being images. Images are unevenly distributed on the manifold, and our task is to devise ways to model this distribution as a probability distribution. We therefore consider popular generative models. For our purposes, generative/probabilistic models should have the properties of 1) sample generation: the possibility to sample from this distribution with the modelled density function, and 2) probability computation: given a previously unseen sample from the dataset of interest, one should be able to compute its probability, at least up to a normalising constant. To this end, we investigate the use of methods such as normalising flow and diffusion models. We then show how semantic interpretations are used to describe points on the manifold. To achieve this, we consider an emergent language framework that uses variational encoders for a disentangled representation of points that reside on a given manifold. Trajectories between points on a manifold can then be described as evolving semantic descriptions. We also show that such probabilistic descriptions (bounded) can be used to improve semantic consistency by constructing defences against adversarial attacks. We evaluate our methods with improved semantic robustness and OoD detection capability, explainable and editable semantic interpolation, and improved classification accuracy under patch attacks. We also discuss the limitation in diffusion models.

Convergence of the Time Discrete Metamorphosis Model on Hadamard Manifolds

A generalized model for optimal transport of images including dissipation and density modulation

Deep Manifold Computing and Visualization

Automating Image Morphing Using Structural Similarity on a Halfway Domain

Monotone vector fields and the proximal point algorithm on Hadamard manifolds

Convergence of spectral discretization for the flow of diffeomorphisms

Space-time metamorphosis

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

Convergence analysis of inexact proximal point algorithms on Hadamard manifolds

Resolvents of Set-Valued Monotone Vector Fields in Hadamard Manifolds

Sliced-Wasserstein Distances and Flows on Cartan-Hadamard Manifolds

Stochastic Metamorphosis in Imaging Science

Progressive Mesh Metamorphosis: Animating Geometrical Models

The Euler-Poincare theory of Metamorphosis

Metamorphosis of Images in Reproducing Kernel Hilbert Spaces

Varying Manifolds in Diffusion: From Time-varying Geometries to Visual Saliency

Convergence of denoising diffusion models under the manifold hypothesis

Modelling Character Motions on Infinite-Dimensional Manifolds

Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions

Probabilistic and Semantic Descriptions of Image Manifolds and Their Applications

A Convergence Rate for Manifold Neural Networks