Abstract:The sparse modeling is an evident manifestation capturing the parsimony principle just described, and sparse models are widespread in statistics, physics, information sciences, neuroscience, computational mathematics, and so on. In statistics the many applications of sparse modeling span regression, classification tasks, graphical model selection, sparse M-estimators and sparse dimensionality reduction. It is also particularly effective in many statistical and machine learning areas where the primary goal is to discover predictive patterns from data which would enhance our understanding and control of underlying physical, biological, and other natural processes, beyond just building accurate outcome black-box predictors. Common examples include selecting biomarkers in biological procedures, finding relevant brain activity locations which are predictive about brain states and processes based on fMRI data, and identifying network bottlenecks best explaining end-to-end performance. Moreover, the research and applications of efficient recovery of high-dimensional sparse signals from a relatively small number of observations, which is the main focus of compressed sensing or compressive sensing, have rapidly grown and became an extremely intense area of study beyond classical signal processing. Likewise interestingly, sparse modeling is directly related to various artificial vision tasks, such as image denoising, segmentation, restoration and superresolution, object or face detection and recognition in visual scenes, and action recognition. In this manuscript, we provide a brief introduction of the basic theory underlying sparse representation and compressive sensing, and then discuss some methods for recovering sparse solutions to optimization problems in effective way, together with some applications of sparse recovery in a machine learning problem known as sparse dictionary learning.

Statistical learning with sparsity

Sparse Models for Machine Learning

An Overview of Computational Sparse Models and Their Applications in Artificial Intelligence

Efficient Probabilistic Latent Semantic Analysis with Sparsity Control

Conditional Sparse Linear Regression

A Survey of Estimation Methods for Sparse High-dimensional Time Series Models

Sparse Modeling for Image and Vision Processing

Rethinking Bayesian Learning for Data Analysis: The Art of Prior and Inference in Sparsity-Aware Modeling

Scalable Sparse Regression for Model Discovery: The Fast Lane to Insight

Slowly Varying Regression under Sparsity

On The Sparse Bayesian Learning Of Linear Models

Nonsparse learning with latent variables

Sparse Unit-Sum Regression

Statistical learning by sparse deep neural networks

Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researchers

Bayesian High-dimensional Linear Regression with Sparse Projection-posterior

Reliable and Efficient Inference of Bayesian Networks from Sparse Data by Statistical Learning Theory

Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks

Sparse representations, inference and learning

Selective Inference for Group-Sparse Linear Models

Nonparametric augmented probability weighting with sparsity