Abstract:Regression mixture models are widely studied in statistics, machine learning and data analysis. Fitting regression mixtures is challenging and is usually performed by maximum likelihood by using the expectation-maximization (EM) algorithm. However, it is well-known that the initialization is crucial for EM. If the initialization is inappropriately performed, the EM algorithm may lead to unsatisfactory results. The EM algorithm also requires the number of clusters to be given a priori; the problem of selecting the number of mixture components requires using model selection criteria to choose one from a set of pre-estimated candidate models. We propose a new fully unsupervised algorithm to learn regression mixture models with unknown number of components. The developed unsupervised learning approach consists in a penalized maximum likelihood estimation carried out by a robust expectation-maximization (EM) algorithm for fitting polynomial, spline and B-spline regressions mixtures. The proposed learning approach is fully unsupervised: 1) it simultaneously infers the model parameters and the optimal number of the regression mixture components from the data as the learning proceeds, rather than in a two-fold scheme as in standard model-based clustering using afterward model selection criteria, and 2) it does not require accurate initialization unlike the standard EM for regression mixtures. The developed approach is applied to curve clustering problems. Numerical experiments on simulated data show that the proposed robust EM algorithm performs well and provides accurate results in terms of robustness with regard initialization and retrieving the optimal partition with the actual number of clusters. An application to real data in the framework of functional data clustering, confirms the benefit of the proposed approach for practical applications.

Supervised learning from incomplete data via an EM approach

Maximum Likelihood from Incomplete Data Via the EM Algorithm

Statistical Machine Learning EM algorithm Lecture Notes 13 : EM algorithm

Big Learning Expectation Maximization

Unsupervised Learning of Dirichlet Process Mixture Models with Missing Data

Flexible High-Dimensional Unsupervised Learning with Missing Data

Unsupervised learning of regression mixture models with unknown number of components

Learning Mixtures of Experts with EM

A Robust and Flexible EM Algorithm for Mixtures of Elliptical Distributions with Missing Data

On Weight Design of Maximum Weighted Likelihood and an Extended EM Algorithm

On the Behavior of the Expectation-Maximization Algorithm for Mixture Models

EM for Mixture of Linear Regression with Clustered Data

On Convergence Properties of the EM Algorithm for Gaussian Mixtures.

Unsupervised learning of finite mixtures using entropy regularization and its application to image segmentation

Learning Active Basis Models by EM-Type Algorithms

Maximum-Likelihood-Estimation Via the Ecm Algorithm - A General Framework

Discriminative unsupervised learning of structured predictors

Expectation-maximization for logistic regression

Federated Expectation Maximization with heterogeneity mitigation and variance reduction

Reinforced mixture learning

Regularized EM algorithm