Abstract:We present a lightweight approach to sequence classification using Ensemble Methods for Hidden Markov Models (HMMs). HMMs offer significant advantages in scenarios with imbalanced or smaller datasets due to their simplicity, interpretability, and efficiency. These models are particularly effective in domains such as finance and biology, where traditional methods struggle with high feature dimensionality and varied sequence lengths. Our ensemble-based scoring method enables the comparison of sequences of any length and improves performance on imbalanced datasets. This study focuses on the binary classification problem, particularly in scenarios with data imbalance, where the negative class is the majority (e.g., normal data) and the positive class is the minority (e.g., anomalous data), often with extreme distribution skews. We propose a novel training approach for HMM Ensembles that generalizes to multi-class problems and supports classification and anomaly detection. Our method fits class-specific groups of diverse models using random data subsets, and compares likelihoods across classes to produce composite scores, achieving high average precisions and AUCs. In addition, we compare our approach with neural network-based methods such as Convolutional Neural Networks (CNNs) and Long Short-Term Memory networks (LSTMs), highlighting the efficiency and robustness of HMMs in data-scarce environments. Motivated by real-world use cases, our method demonstrates robust performance across various benchmarks, offering a flexible framework for diverse applications.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve **the data imbalance problem in sequence classification tasks**, especially in extremely imbalanced datasets (such as in anomaly detection scenarios). Specifically, the author proposes an ensemble method based on Hidden Markov Models (HMMs) to improve the classification performance on imbalanced datasets. #### Main problems: 1. **Data imbalance**: - In real - world datasets, the positive class (such as abnormal events) is usually far less than the negative class (such as normal events). This imbalance will cause the model to be biased towards the majority class, resulting in poor performance on the minority class. - For example, in the financial field, the number of fraudulent transactions is far less than that of normal transactions; in system monitoring, abnormal behaviors are far less than normal operations. 2. **Sequence length differences**: - The lengths of sequence data may vary, which makes it difficult to directly compare the likelihood values of sequences of different lengths. - Traditional HMMs face challenges when dealing with sequences of different lengths because the likelihood value is affected by the sequence length. 3. **High - dimensional features and complexity**: - Sequence data usually has high - dimensional features and complex temporal dependency relationships, which place higher requirements on the modeling ability of the model. - Although existing deep - learning methods can capture complex temporal dependency relationships, they are prone to over - fitting in the case of scarce data and have poor interpretability. #### Solutions: - **HMM Ensemble Method (HMM - e)**: - Train multiple HMM models (each model is based on a random subset of data) to form an ensemble model. - These models can capture different patterns and behaviors, thereby improving the overall classification performance. - Propose a new scoring mechanism for comparing sequences of different lengths and generating a comprehensive score. - **Model diversity**: - Ensure that each sub - model uses different initial parameters and data subsets during the training process to avoid the generation of redundant models. - Adjust hyper - parameters (such as the number of models N, subset factor s) to ensure model diversity. - **Downstream classifier**: - Use the likelihood values generated by the HMM ensemble model as features and input them into downstream classifiers such as Support Vector Machines (SVM) or Neural Networks (NN) to further improve the classification effect. #### Experimental verification: - The author conducted experiments on multiple publicly available genomic benchmark datasets to verify the effectiveness of the proposed method. - The experimental results show that the HMM ensemble method performs well in dealing with imbalanced datasets, especially outperforming single HMMs and other deep - learning methods in terms of AUC - ROC and Average Precision (AP) metrics. In summary, this paper solves the common data imbalance problem in sequence classification tasks by introducing the HMM ensemble method and provides an efficient and robust solution.

Ensemble Methods for Sequence Classification with Hidden Markov Models

Mixture Hidden Markov Models for Sequence Data: The seqHMM Package in R

DBHC: Discrete Bayesian HMM Clustering

A Hidden Markov Ensemble Algorithm Design for Time Series Analysis

State-Sharing Sparse Hidden Markov Models for Personalized Sequences

Introducing the Hidden Neural Markov Chain framework

An Adaptive Online HDP-HMM for Segmentation and Classification of Sequential Data

Hierarchical Ensemble Support Cluster Machine

Characterizing A Database of Sequential Behaviors with Latent Dirichlet Hidden Markov Models

Self-paced Ensemble for Highly Imbalanced Massive Data Classification

DenseHMM: Learning Hidden Markov Models by Learning Dense Representations

Density Ratio Hidden Markov Models

Probabilistic SVM classifier ensemble selection based on GMDH-type neural network

Higher-order Markov models for metagenomic sequence classification

Enhanced Bayesian Gaussian hidden Markov mixture clustering for improved knowledge discovery

Music Sequence Prediction with Mixture Hidden Markov Models

Seq-HyGAN: Sequence Classification via Hypergraph Attention Network

Spectral Learning of Binomial HMMs for DNA Methylation Data

Hidden Markov Model Variants and their Application

Diversified Hidden Markov Models for Sequential Labeling

Hidden Markov models with multivariate bounded asymmetric student's t-mixture model emissions