Abstract:The Hidden Markov Model (HMM) is a crucial probabilistic modeling technique for sequence data processing and statistical learning that has been extensively utilized in various engineering applications. Traditionally, the EM algorithm is employed to fit HMMs, but currently, academics and professionals exhibit augmenting enthusiasm in Bayesian inference. In the Bayesian context, Markov Chain Monte Carlo (MCMC) methods are commonly used for inferring HMMs, but they can be computationally demanding for high-dimensional covariate data. As a rapid substitute, variational approximation has become a noteworthy and effective approximate inference approach, particularly in recent years, for representation learning in deep generative models. However, there has been limited exploration of variational inference for HMMs with high-dimensional covariates. In this article, we develop a mean-field Variational Bayesian method with the double-exponential shrinkage prior to fit high-dimensional HMMs whose hidden states are of discrete types. The proposed method offers the advantage of fitting the model and investigating specific factors that impact the response variable changes simultaneously. In addition, since the proposed method is based on the Variational Bayesian framework, the proposed method can avoid huge memory and intensive computational cost typical of traditional Bayesian methods. In the simulation studies, we demonstrate that the proposed method can quickly and accurately estimate the posterior distributions of the parameters with good performance. We analyzed the Beijing Multi-Site Air-Quality data and predicted the PM2.5 values via the fitted HMMs.

Variational Bayesian Inference in High-Dimensional Linear Mixed Models

Bayesian variable selection in linear regression models with instrumental variables

A Novel Variational Bayesian Method for Variable Selection in Logistic Regression Models.

Variational Inference and Sparsity in High-Dimensional Deep Gaussian Mixture Models

A Variational Approach for Modeling High-dimensional Spatial Generalized Linear Mixed Models

Variational Inference for Large Bayesian Vector Autoregressions

Fast Bayesian inference in a class of sparse linear mixed effects models

Variational Bayesian Variable Selection for High-Dimensional Hidden Markov Models

Dynamic sparsity on dynamic regression models

High-dimensional copula variational approximation through transformation

Bayesian High-dimensional Linear Regression with Sparse Projection-posterior

Empirical Bayes inference in sparse high-dimensional generalized linear models

Bayesian Variable Selection in Linear Regression in One Pass for Large Data Sets

BIVAS: A scalable Bayesian method for bi-level variable selection with applications

Advances in Bayesian model selection consistency for high-dimensional generalized linear models

Stochastic variational inference for large-scale discrete choice models using adaptive batch sizes

Variable Selection for High-dimensional Generalized Linear Models using an Iterated Conditional Modes/Medians Algorithm

A variational Bayes approach to debiased inference for low-dimensional parameters in high-dimensional linear regression

Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selection

Bayesian Grouping-Gibbs Sampling Estimation of High-dimensional Linear Model with Non-sparsity

Adversarial Variational Bayes Methods For Tweedie Compound Poisson Mixed Models