Abstract:Inference for Variational Autoencoders (VAEs) consists of learning two models: (1) a generative model, which transforms a simple distribution over a latent space into the distribution over observed data, and (2) an inference model, which approximates the posterior of the latent codes given data. The two components are learned jointly via a lower bound to the generative model's log marginal likelihood. In early phases of joint training, the inference model poorly approximates the latent code posteriors. Recent work showed that this leads optimization to get stuck in local optima, negatively impacting the learned generative model. As such, recent work suggests ensuring a high-quality inference model via iterative training: maximizing the objective function relative to the inference model before every update to the generative model. Unfortunately, iterative training is inefficient, requiring heuristic criteria for reverting from iterative to joint training for speed. Here, we suggest an inference method that trains the generative and inference models independently. It approximates the posterior of the true model a priori; fixing this posterior approximation, we then maximize the lower bound relative to only the generative model. By conventional wisdom, this approach should rely on the true prior and likelihood of the true model to approximate its posterior (which are unknown). However, we show that we can compute a deterministic, model-agnostic posterior approximation (MAPA) of the true model's posterior. We then use MAPA to develop a proof-of-concept inference method. We present preliminary results on low-dimensional synthetic data that (1) MAPA captures the trend of the true posterior, and (2) our MAPA-based inference performs better density estimation with less computation than baselines. Lastly, we present a roadmap for scaling the MAPA-based inference method to high-dimensional data.

Learning from Citation: Interpretable Patent Valuation via Conditional Variational Autoencoder

On Predictive Patent Valuation: Forecasting Patent Citations and Their Types.

Probabilistic graph-based valuation model for measuring the relative patent value in a valuation scenario

Discovering the realistic paths towards the realization of patent valuation from technical perspectives: defense, implementation or transfer

Patent Citation Dynamics Modeling via Multi-Attention Recurrent Networks

Asset Pricing Via the Conditional Quantile Variational Autoencoder

Marketable value estimation of patents using ensemble learning methodology: Focusing on U.S. patents for the electricity sector

Patent Valuation Using Citations: A Review and Sensitivity Analysis

Learning Correlated Latent Representations with Adaptive Priors

Deep learning-based citation recommendation system for patents

Learning conditional variational autoencoders with missing covariates

InnoVAE: Generative AI for Understanding Patents and Innovation

Patent Litigation Prediction: A Convolutional Tensor Factorization Approach

Towards Model-Agnostic Posterior Approximation for Fast and Accurate Variational Autoencoders

Causal Recurrent Variational Autoencoder for Medical Time Series Generation

Modeling conditional distributions of neural and behavioral data with masked variational autoencoders

PTNS: patent citation trajectory prediction based on temporal network snapshots

Generating Long Financial Report using Conditional Variational Autoencoders with Knowledge Distillation

Early screening of potential breakthrough technologies with enhanced interpretability: A patent-specific hierarchical attention network model

Inference for Conditional Value-at-risk of a Predictive Regression

Patent2Vec: Multi-view representation learning on patent-graphs for patent classification