Abstract:This paper begins with a description of methods for estimating image probability density functions that reflects the observation that such data is usually constrained to lie in restricted regions of the high-dimensional image space-not every pattern of pixels is an image. It is common to say that images lie on a lower-dimensional manifold in the high-dimensional space. However, it is not the case that all points on the manifold have an equal probability of being images. Images are unevenly distributed on the manifold, and our task is to devise ways to model this distribution as a probability distribution. We therefore consider popular generative models. For our purposes, generative/probabilistic models should have the properties of 1) sample generation: the possibility to sample from this distribution with the modelled density function, and 2) probability computation: given a previously unseen sample from the dataset of interest, one should be able to compute its probability, at least up to a normalising constant. To this end, we investigate the use of methods such as normalising flow and diffusion models. We then show how semantic interpretations are used to describe points on the manifold. To achieve this, we consider an emergent language framework that uses variational encoders for a disentangled representation of points that reside on a given manifold. Trajectories between points on a manifold can then be described as evolving semantic descriptions. We also show that such probabilistic descriptions (bounded) can be used to improve semantic consistency by constructing defences against adversarial attacks. We evaluate our methods with improved semantic robustness and OoD detection capability, explainable and editable semantic interpolation, and improved classification accuracy under patch attacks. We also discuss the limitation in diffusion models.

On the Construction of Joint Probabilistic Image Models

Probabilistic Image Model and Its Information-Theoretic Properties

Learning structural conjunction of image content by sparse graphical model

Probabilistic and Semantic Descriptions of Image Manifolds and Their Applications

Joint Distortion Model for Progressive Image Transmission Using Error Correcting Arithmetic Codes

A Nonparametric Bayesian Approach to Image Compressive Sensing on Manifolds with Correlation Constraints

Joint Reconstruction of Multi-view Compressed Images

A Methodology for Modeling the Distributions of Medical Images and Their Stochastic Properties

End-to-End Image Compression with Probabilistic Decoding

Joint Reconstruction of Multiview Compressed Images

Robust Joint Reconstruction in Compressed Multi-View Imaging.

Joint Estimation of Image Representations and their Lie Invariants

Joint Global and Local Hierarchical Priors for Learned Image Compression

Probabilistic Local Structure Models

Image Restoration Using Joint Statistical Modeling in a Space-Transform Domain

Context Probability Modeling for Encoding Quantized Transform Coefficients

On Modeling the Distribution of Chest X-ray Images

Correlation Model for A Class of Medical Images

Probabilistic models, compressible interactions, and neural coding

Distributed image coding based on integrated Markov random field modeling and LDPC decoding

Bayesian learning of joint distributions of objects