Abstract:In this paper, we propose a novel method of model-based time series clustering with mixtures of general state space models (MSSMs). Each component of MSSMs is associated with each cluster. An advantage of the proposed method is that it enables the use of time series models appropriate to the specific time series. This not only improves clustering and prediction accuracy but also enhances the interpretability of the estimated parameters. The parameters of the MSSMs are estimated using stochastic variational inference, a subtype of variational inference. The proposed method estimates the latent variables of an arbitrary state space model by using neural networks with a normalizing flow as a variational estimator. The number of clusters can be estimated using the Bayesian information criterion. In addition, to prevent MSSMs from converging to the local optimum, we propose several optimization tricks, including an additional penalty term called entropy annealing. To our best knowledge, the proposed method is the first computationally feasible one for time series clustering based on general (possibly nonlinear, non-Gaussian) state space models. Experiments on simulated datasets show that the proposed method is effective for clustering, parameter estimation, and estimating the number of clusters.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the model selection and parameter estimation in time - series clustering. Specifically, the author proposes a new time - series clustering method based on Mixture of General State Space Models (MSSMs), aiming to improve the accuracy of clustering and prediction and enhance the interpretability of the estimated parameters at the same time. ### Problem Background Time - series clustering is a technique used to classify multiple time - series data into groups with similar patterns. Traditional clustering methods include shape - based, feature - based and model - based methods. Among them, model - based methods can provide higher accuracy and better interpretability by using appropriate prediction models. However, existing model - based methods (such as AR models, HMMs and LGSSMs) have limitations when dealing with non - linear, non - Gaussian dynamics and non - stationary time - series. ### Method Proposed in the Paper To solve the above problems, this paper proposes a time - series clustering method based on Mixture of General State Space Models (MSSMs). The main features of this method are as follows: 1. **Flexibility**: It allows the use of arbitrary state and observation equations, so it can describe the dynamic characteristics of time - series more flexibly. 2. **Expressive Power**: It introduces Stochastic Variational Inference (SVI) in variational inference and combines it with normalizing flows to improve the expressive power of the approximate distribution. 3. **Optimization Techniques**: To prevent the model from falling into local optimal solutions, optimization techniques such as entropy annealing are proposed. 4. **Cluster Number Estimation**: It uses the Bayesian Information Criterion (BIC) to estimate the number of clusters. ### Advantages of the Method - **Improve Clustering and Prediction Accuracy**: By using models suitable for specific time - series, the accuracy of clustering and prediction is improved. - **Enhance Parameter Interpretability**: It can better interpret the estimated parameters, thus providing more insights. - **Computational Feasibility**: For the first time, the computational feasibility of time - series clustering based on general state space models is achieved. ### Experimental Results Through experiments on simulated data sets, the effectiveness of this method in clustering, parameter estimation and cluster number estimation is verified. The experimental results show that this method can accurately estimate model parameters while correctly identifying the number of clusters, and can fit latent variables well. In conclusion, this paper aims to solve the limitations of existing methods in dealing with complex time - series by proposing a new time - series clustering method based on Mixture of General State Space Models, thereby improving the performance of clustering and prediction.

Time Series Clustering with General State Space Models via Stochastic Variational Inference

Time Series Clustering with an EM algorithm for Mixtures of Linear Gaussian State Space Models

Relational State-Space Model for Stochastic Multi-Object Systems

Model-based clustering and classification using mixtures of multivariate skewed power exponential distributions

Clustering of non-Gaussian data by variational Bayes for normal inverse Gaussian mixture models

Bayesian mixtures of common factor analyzers: Model, variational inference, and applications

Time-Varying Gaussian Markov Random Fields Learning for Multivariate Time Series Clustering

Variational learning for switching state-space models

Stochastic Gradient MCMC for Nonlinear State Space Models

Clustering Time Series Data with Gaussian Mixture Embeddings in a Graph Autoencoder Framework

Stanza: A Nonlinear State Space Model for Probabilistic Inference in Non-Stationary Time Series

Particle Gibbs for Likelihood-Free Inference of State Space Models with Application to Stochastic Volatility

The parsimonious Gaussian mixture models with partitioned parameters and their application in clustering

Contrastive learning-based multi-view clustering for incomplete multivariate time series

Scalable Hybrid HMM with Gaussian Process Emission for Sequential Time-series Data Clustering

Variational Bayes Approximations for Clustering via Mixtures of Normal Inverse Gaussian Distributions

Unsupervised Clustering Through Gaussian Mixture Variational AutoEncoder with Non-Reparameterized Variational Inference and Std Annealing.

Simultaneous Bayesian Clustering and Model Selection with Mixture of Robust Factor Analyzers

Coupling LSTM neural networks and state-space models through analytically tractable inference

Variational learning of hierarchical infinite generalized Dirichlet mixture models and applications

Model-based Clustering with Sparse Covariance Matrices