Abstract:Sparse modeling for signal processing and machine learning has been at the focus of scientific research for over two decades. Among others, supervised sparsity-aware learning comprises two major paths paved by: a) discriminative methods and b) generative methods. The latter, more widely known as Bayesian methods, enable uncertainty evaluation w.r.t. the performed predictions. Furthermore, they can better exploit related prior information and naturally introduce robustness into the model, due to their unique capacity to marginalize out uncertainties related to the parameter estimates. Moreover, hyper-parameters associated with the adopted priors can be learnt via the training data. To implement sparsity-aware learning, the crucial point lies in the choice of the function regularizer for discriminative methods and the choice of the prior distribution for Bayesian learning. Over the last decade or so, due to the intense research on deep learning, emphasis has been put on discriminative techniques. However, a come back of Bayesian methods is taking place that sheds new light on the design of deep neural networks, which also establish firm links with Bayesian models and inspire new paths for unsupervised learning, such as Bayesian tensor decomposition. The goal of this article is two-fold. First, to review, in a unified way, some recent advances in incorporating sparsity-promoting priors into three highly popular data modeling tools, namely deep neural networks, Gaussian processes, and tensor decomposition. Second, to review their associated inference techniques from different aspects, including: evidence maximization via optimization and variational inference methods. Challenges such as small data dilemma, automatic model structure search, and natural prediction uncertainty evaluation are also discussed. Typical signal processing and machine learning tasks are demonstrated.

Sparsity enforcing priors in inverse problems via Normal variance mixtures: model selection, algorithms and applications

Fast Bayesian hierarchical inference via sparsity enforcing a priori for aeroacoustic source imaging

Generalized sparsity-promoting solvers for Bayesian inverse problems: Versatile sparsifying transforms and unknown noise variances

A Variational Bayesian Approximation Approach Via A Sparsity Enforcing Prior In Acoustic Imaging

Sparse reconstructions from few noisy data: analysis of hierarchical Bayesian models with generalized gamma hyperpriors

Learning a Gaussian Mixture for Sparsity Regularization in Inverse Problems

Sparsity promoting hybrid solvers for hierarchical Bayesian inverse problems

Posterior contraction rates in a sparse non-linear mixed-effects model

Bayesian High-dimensional Linear Regression with Sparse Projection-posterior

Sparse bayesian inference with regularized gaussian distributions

Tackling the Problem of Distributional Shifts: Correcting Misspecified, High-Dimensional Data-Driven Priors for Inverse Problems

Bayesian inversion with Student's t priors based on Gaussian scale mixtures

Computationally Efficient Sampling Methods for Sparsity Promoting Hierarchical Bayesian Models

Prior normalization for certified likelihood-informed subspace detection of Bayesian inverse problems

Adaptive posterior concentration rates for sparse high-dimensional linear regression with random design and unknown error variance

High-dimensional Grouped-regression using Bayesian Sparse Projection-posterior

A posterior contraction for Bayesian inverse problems in Banach spaces

Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modeling

EP-GIG Priors and Applications in Bayesian Sparse Learning

Generalized Bayes Approach to Inverse Problems with Model Misspecification

Bayesian approach to inverse scattering with topological priors