Abstract:Sparse approximation is key to many signal processing, image processing, and machine learning applications. If multiple signals maintain some degree of dependency, for example, the support sets are statistically related, then it will generally be advantageous to jointly estimate the sparse representation vectors from the measurement vectors as opposed to solving for each signal individually. In this paper, we propose simultaneous sparse Bayesian learning (SBL) for joint sparse approximation with two structured sparse models (SSMs), where one is row-sparse with embedded element-sparse and the other one is row-sparse plus element-sparse. While SBL has attracted much attention as a means to deal with a single sparse approximation problem, it is not obvious how to extend SBL to SSMs. By capitalizing on a dual-space view of existing convex methods for SMs, we showcase the precision component model and covariance component model for SSMs, where both models involve a common hyperparameter and an innovation hyperparameter that together control the prior variance for each coefficient. The statistical perspective of precision component versus covariance component models unfolds the intrinsic mechanism in SSMs, and also leads to our development of SBL-inspired cost functions for SSMs. Centralized algorithms that include 11 and 12 reweighting algorithms and consensus-based decentralized algorithms are developed for simultaneous sparse approximation with SSMs. In addition, theoretical analysis is conducted to provide valuable insights into the proposed approach, which includes global minima analysis of the SBL-inspired nonconvex cost functions and convergence analysis of the proposed 11 reweighting algorithms for SSMs. Superior performance of the proposed algorithms is demonstrated by numerical experiments.

A Multi-Resolution Approach with Sparseness Property for Input-Output Approximation

A Variational Bayesian Approximation Approach Via A Sparsity Enforcing Prior In Acoustic Imaging

Geophysical Data Sparse Reconstruction Based on L0-Norm Minimization

Surface Approximation Via Sparse Representation and Parameterization Optimization

Linear Programming-Based Sparse Kernel Regression with L1-Norm Minimization for Nonlinear System Modeling

A Fast Super-Resolution Method Based on Sparsity Properties.

AMS-Net: Adaptive Multiscale Sparse Neural Network with Interpretable Basis Expansion for Multiphase Flow Problems

Simultaneous Bayesian Sparse Approximation with Structured Sparse Models

A self-learning image super-resolution method via sparse representation and non-local similarity

Operator learning based on sparse high-dimensional approximation

A gradient based technique for generating sparse representation in function approximation

A novel approach for fusion of heterogeneous sources of data

Sparse approximation of multilinear problems with applications to kernel-based methods in UQ

An efficient active-set method with applications to sparse approximations and risk minimization

Augmented Flexible Krylov Subspace methods with applications to Bayesian inverse problems

Pareto-based Optimization of Sparse Dynamical Systems

Joint State and Sparse Input Estimation in Linear Dynamical Systems

A SUCCESSIVE CONVEX APPROXIMATION APPROACH FOR SPARSE SOLUTIONS OF CONVEX PROGRAMS

Sparse successive approximation for nonlinear H2 and H∞ optimal control problems under residual errors

Minimization Over the Nonconvex Sparsity Constraint Using A Hybrid First-order method

A multi-resolution, non-parametric, Bayesian framework for identification of spatially-varying model parameters