Abstract:Hidden Markov Models (HMM) model a sequence of observations that are dependent on a hidden (or latent) state that follow a Markov chain. These models are widely used in diverse fields including ecology, speech recognition, and genetics.Parameter estimation in HMM is typically performed using the Baum-Welch algorithm, a special case of the Expectation-Maximisation (EM) algorithm. While this method guarantee the convergence to a local maximum, its convergence rates is usually slow.Alternative methods, such as the direct maximisation of the likelihood using quasi-Newton methods (such as L-BFGS-B) can offer faster convergence but can be more complicated to implement due to challenges to deal with the presence of bounds on the space of parameters.We propose a novel hybrid algorithm, QNEM, that combines the Baum-Welch and the quasi-Newton algorithms. QNEM aims to leverage the strength of both algorithms by switching from one method to the other based on the convexity of the likelihood function.We conducted a comparative analysis between QNEM, the Baum-Welch algorithm, an EM acceleration algorithm called SQUAREM (Varadhan, 2008, Scand J Statist), and the L-BFGS-B quasi-Newton method by applying these algorithms to four examples built on different models. We estimated the parameters of each model using the different algorithms and evaluated their performances.Our results show that the best-performing algorithm depends on the model considered. QNEM performs well overall, always being faster or equivalent to L-BFGS-B. The Baum-Welch and SQUAREM algorithms are faster than the quasi-Newton and QNEM algorithms in certain scenarios with multiple optimum. In conclusion, QNEM offers a promising alternative to existing algorithms.

Hidden Markov Mixture Autoregressive Models: Parameter Estimation

Bayesian estimation and prediction for certain mixtures

Maximum-penalized-likelihood estimation for independent and Markov-dependent mixture models.

Bayesian mixtures of common factor analyzers: Model, variational inference, and applications

Hidden Markov Model Framework Using Independent Component Analysis Mixture Model

Mixture Matrix-valued Autoregressive Model

Parameter estimation of hidden Markov models: comparison of EM and quasi-Newton methods with a new hybrid algorithm

Multimode process data modeling: A Dirichlet process mixture model based Bayesian robust factor analyzer approach

Mixture Bayesian Regularization of PCR Model and Soft Sensing Application

Parameter Estimation for Hidden Markov Models with Intractable Likelihoods

On the Behavior of the Expectation-Maximization Algorithm for Mixture Models

Estimating mixed-effects state-space models via particle filters and the EM algorithm

Towards interpretability of Mixtures of Hidden Markov Models

A Nonparametric Estimation Method for the Multivariate Mixture Models

Estimation and Hypothesis-Testing in Finite Mixture-Models

Parameter estimation for stable distributions and their mixture

Mixture of hidden Markov models for accelerometer data

Parameter Estimation of Poisson Mixture with Automated Model Selection Through BYY Harmony Learning

Time series clustering based on latent volatility mixture modeling with applications in finance

Hidden Markov models with multivariate bounded asymmetric student's t-mixture model emissions

An Mcmc Based Em Algorithm For Mixtures Of Gaussian Processes