Abstract:Sparse approximation is key to many signal processing, image processing, and machine learning applications. If multiple signals maintain some degree of dependency, for example, the support sets are statistically related, then it will generally be advantageous to jointly estimate the sparse representation vectors from the measurement vectors as opposed to solving for each signal individually. In this paper, we propose simultaneous sparse Bayesian learning (SBL) for joint sparse approximation with two structured sparse models (SSMs), where one is row-sparse with embedded element-sparse and the other one is row-sparse plus element-sparse. While SBL has attracted much attention as a means to deal with a single sparse approximation problem, it is not obvious how to extend SBL to SSMs. By capitalizing on a dual-space view of existing convex methods for SMs, we showcase the precision component model and covariance component model for SSMs, where both models involve a common hyperparameter and an innovation hyperparameter that together control the prior variance for each coefficient. The statistical perspective of precision component versus covariance component models unfolds the intrinsic mechanism in SSMs, and also leads to our development of SBL-inspired cost functions for SSMs. Centralized algorithms that include 11 and 12 reweighting algorithms and consensus-based decentralized algorithms are developed for simultaneous sparse approximation with SSMs. In addition, theoretical analysis is conducted to provide valuable insights into the proposed approach, which includes global minima analysis of the SBL-inspired nonconvex cost functions and convergence analysis of the proposed 11 reweighting algorithms for SSMs. Superior performance of the proposed algorithms is demonstrated by numerical experiments.

Stochastic Ml Simplex-Structured Matrix Factorization Under The Dirichlet Mixture Model

The Infinite Student'S T-Mixture For Robust Modeling

Bayesian mixtures of common factor analyzers: Model, variational inference, and applications

Probabilistic Matrix Factorization for Data with Attributes Based on Finite Mixture Modeling

Dual Simplex Volume Maximization for Simplex-Structured Matrix Factorization

Distributed Density Estimation Based on a Mixture of Factor Analyzers in a Sensor Network.

Semi-supervised Mixture of Latent Factor Analysis Models with Application to Online Key Variable Estimation

Multimode process data modeling: A Dirichlet process mixture model based Bayesian robust factor analyzer approach

Efficient Probabilistic Latent Semantic Analysis with Sparsity Control

Mixture Rank Matrix Factorization Model*

Infinite max-margin factor analysis via data augmentation

Model-based clustering via mixtures of unrestricted skew normal factor analyzers with complete and incomplete data

A Sparse Component Analysis Algorithm Based on Finite-Mixture-Model Learning

Simultaneous Bayesian Clustering and Model Selection with Mixture of Robust Factor Analyzers

A Nonparametric Estimation Method for the Multivariate Mixture Models

Simultaneous Bayesian Sparse Approximation with Structured Sparse Models

A novel finite mixture model based on the generalized scale mixtures of asymmetric generalized normal distributions: properties, estimation methodology and applications

FiMReSt: Finite Mixture of Multivariate Regulated Skew-t Kernels -- A Flexible Probabilistic Model for Multi-Clustered Data with Asymmetrically-Scattered Non-Gaussian Kernels

@@@@Sparse Constrained Manifold Regularized Concept Factorization Algorithm

Robust Classification via Finite Mixtures of Matrix Variate Skew-t Distributions

Robust mixture modelling using multivariate