Abstract:Sparse modeling for signal processing and machine learning has been at the focus of scientific research for over two decades. Among others, supervised sparsity-aware learning comprises two major paths paved by: a) discriminative methods and b) generative methods. The latter, more widely known as Bayesian methods, enable uncertainty evaluation w.r.t. the performed predictions. Furthermore, they can better exploit related prior information and naturally introduce robustness into the model, due to their unique capacity to marginalize out uncertainties related to the parameter estimates. Moreover, hyper-parameters associated with the adopted priors can be learnt via the training data. To implement sparsity-aware learning, the crucial point lies in the choice of the function regularizer for discriminative methods and the choice of the prior distribution for Bayesian learning. Over the last decade or so, due to the intense research on deep learning, emphasis has been put on discriminative techniques. However, a come back of Bayesian methods is taking place that sheds new light on the design of deep neural networks, which also establish firm links with Bayesian models and inspire new paths for unsupervised learning, such as Bayesian tensor decomposition. The goal of this article is two-fold. First, to review, in a unified way, some recent advances in incorporating sparsity-promoting priors into three highly popular data modeling tools, namely deep neural networks, Gaussian processes, and tensor decomposition. Second, to review their associated inference techniques from different aspects, including: evidence maximization via optimization and variational inference methods. Challenges such as small data dilemma, automatic model structure search, and natural prediction uncertainty evaluation are also discussed. Typical signal processing and machine learning tasks are demonstrated.

Bayesian Deep Learning for Image Reconstruction: from Structured Sparsity to Uncertainty Estimation.

Bayesian Inference and Deep Learning for Inverse Problems

Sparse Bayesian Learning Approach for Discrete Signal Reconstruction

A Variational Bayesian Approximation Approach Via A Sparsity Enforcing Prior In Acoustic Imaging

Subspace Model-Assisted Deep Learning for Improved Image Reconstruction.

Deep Probabilistic Imaging: Uncertainty Quantification and Multi-modal Solution Characterization for Computational Imaging.

On the Quantification of Image Reconstruction Uncertainty without Training Data

A Low-Complexity Bayesian Approach to Large-Scale Sparse Image Reconstruction with Structured Constraints

Sparse Bayesian Deep Learning for Cross Domain Medical Image Reconstruction

Deep-learning-driven Reliable Single-pixel Imaging with Uncertainty Approximation

Self-supervised Bayesian Deep Learning for Image Recovery with Applications to Compressive Sensing

Large-scale structured sparse image reconstruction with correlated multiple-measurement vectors using Bayesian learning

Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modeling

Joint Segmentation and Image Reconstruction with Error Prediction in Photoacoustic Imaging using Deep Learning

Uncertainty Quantification for Deep Unrolling-Based Computational Imaging

Image Super-resolution Reconstruction Algorithm Based on Bayesian Theory

A Bayesian Based Deep Unrolling Algorithm for Single-Photon Lidar Systems

Bayesian Based Unrolling for Reconstruction and Super-resolution of Single-Photon Lidar Systems

Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification

General Intelligent Imaging and Uncertainty Quantification by Deterministic Diffusion Model

Bayesian Image Super-Resolution with Deep Modeling of Image Statistics.