Abstract:Sparse Bayesian Learning, and more specifically the Relevance Vector Machine (RVM), can be used in supervised learning for both classification and regression problems. Such methods are particularly useful when applied to big data in order to find a sparse (in weight space) representation of the model. This paper demonstrates that the Template Model Builder (TMB) is an accurate and flexible computational framework for implementation of sparse Bayesian learning methods.The user of TMB is only required to specify the joint likelihood of the weights and the data, while the Laplace approximation of the marginal likelihood is automatically evaluated to numerical precision. This approximation is in turn used to estimate hyperparameters by maximum marginal likelihood. In order to reduce the computational cost of the Laplace approximation we introduce the notion of an "active set" of weights, and we devise an algorithm for dynamically updating this set until convergence, similar to what is done in other RVM type methods. We implement two different methods using TMB; the RVM and the Probabilistic Feature Selection and Classification Vector Machine method, where the latter also performs feature selection. Experiments based on benchmark data show that our TMB implementation performs comparable to that of the original implementation, but at a lower implementation cost. TMB can also calculate model and prediction uncertainty, by including estimation uncertainty from both latent variables and the hyperparameters. In conclusion, we find that TMB is a flexible tool that facilitates implementation and prototyping of sparse Bayesian methods.

Heteroscedastic Relevance Vector Machine

Expectile regression for analyzing heteroscedasticity in high dimension

Robust Estimation and Shrinkage in Ultrahigh Dimensional Expectile Regression with Heavy Tails and Variance Heterogeneity

Semiparametric Expectile Regression for High-dimensional Heavy-tailed and Heterogeneous Data

Investigation on the Construction of the Relevance Vector Machine Based on Cross Entropy Minimization

Improved Hyperspectral Land-Cover Analysis Using Relevance Vector Machine

Variational Heteroscedastic Volatility Model

Nonlinear Soft Sensor Development Based on Relevance Vector Machine

On the construction of the relevance vector machine based on Bayesian Ying-Yang harmony learning

Relevance Vector Machine Based on Bayesian Learning and Its Application in Soft Sensing

Sparse Bayesian learning using TMB (Template Model Builder)

Research progress and analysis on methods for classification of RVM

Semi-supervised Variational Bayesian Student’s T Mixture Regression and Robust Inferential Sensor Application

Approximation Properties of Variational Bayes for Vector Autoregressions

Robust Hyperspectral Classification Using Relevance Vector Machine

A Gaussian Approximation of Marginal Likelihood in Relevance Vector Machine for Industrial Data With Input Noise

Combined modeling of sparse and dense noise for improvement of Relevance Vector Machine

Fuzzy Relevance Vector Machine for Learning from Unbalanced Data and Noise

Relevance vector machine for efficient classification of scattered patterns in hyperspectral imagery

Scalable Random Feature Latent Variable Models

Svr Plus Rvr: A Robust Sparse Kernel Method For Regression