Abstract:The Hidden Markov Model (HMM) is a crucial probabilistic modeling technique for sequence data processing and statistical learning that has been extensively utilized in various engineering applications. Traditionally, the EM algorithm is employed to fit HMMs, but currently, academics and professionals exhibit augmenting enthusiasm in Bayesian inference. In the Bayesian context, Markov Chain Monte Carlo (MCMC) methods are commonly used for inferring HMMs, but they can be computationally demanding for high-dimensional covariate data. As a rapid substitute, variational approximation has become a noteworthy and effective approximate inference approach, particularly in recent years, for representation learning in deep generative models. However, there has been limited exploration of variational inference for HMMs with high-dimensional covariates. In this article, we develop a mean-field Variational Bayesian method with the double-exponential shrinkage prior to fit high-dimensional HMMs whose hidden states are of discrete types. The proposed method offers the advantage of fitting the model and investigating specific factors that impact the response variable changes simultaneously. In addition, since the proposed method is based on the Variational Bayesian framework, the proposed method can avoid huge memory and intensive computational cost typical of traditional Bayesian methods. In the simulation studies, we demonstrate that the proposed method can quickly and accurately estimate the posterior distributions of the parameters with good performance. We analyzed the Beijing Multi-Site Air-Quality data and predicted the PM2.5 values via the fitted HMMs.

MM Algorithms for Variance Components Models

An Improved Algorithm for High-Dimensional Continuous Threshold Expectile Model with Variance Heterogeneity

Accelerated Stochastic ADMM with Variance Reduction

A Tutorial on MM Algorithms

EIGENVALUE DISTRIBUTIONS OF VARIANCE COMPONENTS ESTIMATORS IN HIGH-DIMENSIONAL RANDOM EFFECTS MODELS

MM Algorithms for Statistical Estimation in Quantile Regression

Stochastic Variance-Reduced Majorization-Minimization Algorithms

Model Averaging Estimation for Nonparametric Varying-Coefficient Models with Multiplicative Heteroscedasticity

Essential formulae for restricted maximum likelihood and its derivatives associated with the linear mixed models

An Introduction to MM Algorithms for Machine Learning and Statistical

MM for Penalized Estimation

An Approximate Restricted Likelihood Ratio Test for Variance Components in Generalized Linear Mixed Models

Nonconvex Optimization via MM Algorithms: Convergence Theory

Asymptotic Properties of Estimation in Generalized Partial Linear Mixed Models

Partially factorized variational inference for high-dimensional mixed models

Estimating the Number of Components in Finite Mixture Models via Variational Approximation

Variational Bayesian Variable Selection for High-Dimensional Hidden Markov Models

Quadratic Estimates of Variance Components and Admissibility

Variance Component Testing in Semiparametric Mixed Models

Quasi Maximum Likelihood Estimation for Large-Dimensional Matrix Factor Models

A robust and efficient estimation and variable selection method for partially linear models with large-dimensional covariates