Abstract:In this paper we contribute a novel algorithm family, which generalizes many unsupervised techniques including unnormalized and energy models, and allows us to infer different statistical modalities (e.g. data likelihood and ratio between densities) from data samples. The proposed unsupervised technique, named Probabilistic Surface Optimization (PSO), views a model as a flexible surface which can be pushed according to loss-specific virtual stochastic forces, where a dynamical equilibrium is achieved when the pointwise forces on the surface become equal. Concretely, the surface is pushed up and down at points sampled from two different distributions. The averaged up and down forces become functions of these two distribution densities and of force magnitudes defined by the loss of a particular PSO instance. Upon convergence, the force equilibrium imposes an optimized model to be equal to various statistical functions depending on the used magnitude functions. Furthermore, this dynamical-statistical equilibrium is extremely intuitive and useful, providing many implications and possible usages in probabilistic inference. We connect PSO to numerous existing statistical works which are also PSO instances, and derive new PSO-based inference methods as demonstration of PSO exceptional usability. Likewise, based on the insights coming from the virtual-force perspective we analyze PSO stability and propose new ways to improve it. Finally, we present new instances of PSO, termed PSO-LDE, for data log-density estimation and also provide a new NN block-diagonal architecture for increased surface flexibility, which significantly improves estimation accuracy. Both PSO-LDE and the new architecture are combined together as a new density estimation technique. In our experiments we demonstrate this technique to be superior over state-of-the-art baselines in density estimation task for multimodal 20D data.

Deep PDF: Probabilistic Surface Optimization and Density Estimation

General Probabilistic Surface Optimization and Log Density Estimation

Data-driven deep density estimation

Distilling Slow Process Probability Density from Fast Random Data

Kernel Density Matrices for Probabilistic Deep Learning

Error-aware Sampling in Adaptive Shells for Neural Surface Reconstruction

Evolutionary probability density reconstruction of stochastic dynamic responses based on physics-aided deep learning

Deep data density estimation through Donsker-Varadhan representation

Probabilistic Transfer Learning Through Ensemble Probabilistic Deep Neural Network

Inference and Computation for Sparsely Sampled Random Surfaces

PDE-Based Dynamic Density Estimation for Large-Scale Agent Systems

DeepFit: 3D Surface Fitting via Neural Network Weighted Least Squares

Deep Optimized Priors for 3D Shape Modeling and Reconstruction

Deep Optimal Sensor Placement for Black Box Stochastic Simulations

Exploring Probabilistic Distance Fields in Robotics

Probabilistic 3D Surface Reconstruction from Sparse MRI Information

Error Bounds for Deep Learning-based Uncertainty Propagation in SDEs

ODE-DPS: ODE-based Diffusion Posterior Sampling for Inverse Problems in Partial Differential Equation

Analysis of optimization methods for nonparametric estimation of probability density in large volume samples

Deep Physics Corrector: A physics enhanced deep learning architecture for solving stochastic differential equations

Learning PDEs from data on closed surfaces with sparse optimization