Abstract:Sparse Bayesian Learning, and more specifically the Relevance Vector Machine (RVM), can be used in supervised learning for both classification and regression problems. Such methods are particularly useful when applied to big data in order to find a sparse (in weight space) representation of the model. This paper demonstrates that the Template Model Builder (TMB) is an accurate and flexible computational framework for implementation of sparse Bayesian learning methods.The user of TMB is only required to specify the joint likelihood of the weights and the data, while the Laplace approximation of the marginal likelihood is automatically evaluated to numerical precision. This approximation is in turn used to estimate hyperparameters by maximum marginal likelihood. In order to reduce the computational cost of the Laplace approximation we introduce the notion of an "active set" of weights, and we devise an algorithm for dynamically updating this set until convergence, similar to what is done in other RVM type methods. We implement two different methods using TMB; the RVM and the Probabilistic Feature Selection and Classification Vector Machine method, where the latter also performs feature selection. Experiments based on benchmark data show that our TMB implementation performs comparable to that of the original implementation, but at a lower implementation cost. TMB can also calculate model and prediction uncertainty, by including estimation uncertainty from both latent variables and the hyperparameters. In conclusion, we find that TMB is a flexible tool that facilitates implementation and prototyping of sparse Bayesian methods.

jti and sparta: Time and Space Efficient Packages for Model-Based Prediction in Large Bayesian Networks

sparta: Sparse Tables and their Algebra with a View Towards High Dimensional Graphical Models

Reliable and Efficient Inference of Bayesian Networks from Sparse Data by Statistical Learning Theory

RJHMC-Tree for Exploration of the Bayesian Decision Tree Posterior

A modeling framework for detecting and leveraging node-level information in Bayesian network inference

Bayesian modeling of temporal dependence in large sparse contingency tables

Efficient inference in persistent Dynamic Bayesian Networks

A Full Bayesian Approach to Sparse Network Inference Using Heterogeneous Datasets

Sparse Bayesian learning using TMB (Template Model Builder)

Efficient parameter learning for Bayesian Network classifiers following the Apache Spark Dataframes paradigm

Neural Relation Inference for Multi-dimensional Temporal Point Processes Via Message Passing Graph.

Integrated Nested Laplace Approximations for Large-Scale Spatial-Temporal Bayesian Modeling

A Spectral Algorithm for Latent Junction Trees

Sparse network estimation for dynamical spatio-temporal array models

Top-Down Bayesian Posterior Sampling for Sum-Product Networks

Tensor Graphical Model: Non-Convex Optimization and Statistical Inference

Dynamic Chain Graph Models for Ordinal Time Series Data

spBayes for large univariate and multivariate point-referenced spatio-temporal data models

Joint Modeling of Multiple Related Time Series via the Beta Process

Bayesian High-dimensional Linear Regression with Sparse Projection-posterior

Leveraging Joint Sparsity in Hierarchical Bayesian Learning