Abstract:In this paper we contribute a novel algorithm family, which generalizes many unsupervised techniques including unnormalized and energy models, and allows us to infer different statistical modalities (e.g. data likelihood and ratio between densities) from data samples. The proposed unsupervised technique, named Probabilistic Surface Optimization (PSO), views a model as a flexible surface which can be pushed according to loss-specific virtual stochastic forces, where a dynamical equilibrium is achieved when the pointwise forces on the surface become equal. Concretely, the surface is pushed up and down at points sampled from two different distributions. The averaged up and down forces become functions of these two distribution densities and of force magnitudes defined by the loss of a particular PSO instance. Upon convergence, the force equilibrium imposes an optimized model to be equal to various statistical functions depending on the used magnitude functions. Furthermore, this dynamical-statistical equilibrium is extremely intuitive and useful, providing many implications and possible usages in probabilistic inference. We connect PSO to numerous existing statistical works which are also PSO instances, and derive new PSO-based inference methods as demonstration of PSO exceptional usability. Likewise, based on the insights coming from the virtual-force perspective we analyze PSO stability and propose new ways to improve it. Finally, we present new instances of PSO, termed PSO-LDE, for data log-density estimation and also provide a new NN block-diagonal architecture for increased surface flexibility, which significantly improves estimation accuracy. Both PSO-LDE and the new architecture are combined together as a new density estimation technique. In our experiments we demonstrate this technique to be superior over state-of-the-art baselines in density estimation task for multimodal 20D data.

A mixture of ellipsoidal densities for 3D data modelling

Gaussian mixture density modeling and decomposition with weighted likelihood

Model-based clustering and classification using mixtures of multivariate skewed power exponential distributions

A Robust and Flexible EM Algorithm for Mixtures of Elliptical Distributions with Missing Data

Inferring 3D Ellipsoids Based on Cross-Sectional Images with Applications to Porosity Control of Additive Manufacturing

A Bayesian Approach Toward Robust Multidimensional Ellipsoid-Specific Fitting

A robust cylindrical fitting to point cloud data

Approximation of probability density functions on the Euclidean group parametrized by dual quaternions

Positive Data Modeling Using a Mixture of Mixtures of Inverted Beta Distributions.

Moment-based dirac mixture approximation of circular densities

Differentiability and Approximation of Probability Functions under Gaussian Mixture Models: A Bayesian Approach

Density Parameter Estimation for Additive Cauchy-Gaussian Mixture

Flexible Clustering with a Sparse Mixture of Generalized Hyperbolic Distributions

Data-driven regularization of Wasserstein barycenters with an application to multivariate density registration

Sensitivity-Aware Density Estimation in Multiple Dimensions

Nearest Neighbor Dirichlet Mixtures

Estimating a density near an unknown manifold: a Bayesian nonparametric approach

Robust Ellipsoid-specific Fitting via Expectation Maximization.

Learning Probability Density Functions from Marginal Distributions with Applications to Gaussian Mixtures

A Generalized Finite Mixture Model for Asymmetric Probability Distribution Observations

General Probabilistic Surface Optimization and Log Density Estimation