Abstract:The sparse modeling is an evident manifestation capturing the parsimony principle just described, and sparse models are widespread in statistics, physics, information sciences, neuroscience, computational mathematics, and so on. In statistics the many applications of sparse modeling span regression, classification tasks, graphical model selection, sparse M-estimators and sparse dimensionality reduction. It is also particularly effective in many statistical and machine learning areas where the primary goal is to discover predictive patterns from data which would enhance our understanding and control of underlying physical, biological, and other natural processes, beyond just building accurate outcome black-box predictors. Common examples include selecting biomarkers in biological procedures, finding relevant brain activity locations which are predictive about brain states and processes based on fMRI data, and identifying network bottlenecks best explaining end-to-end performance. Moreover, the research and applications of efficient recovery of high-dimensional sparse signals from a relatively small number of observations, which is the main focus of compressed sensing or compressive sensing, have rapidly grown and became an extremely intense area of study beyond classical signal processing. Likewise interestingly, sparse modeling is directly related to various artificial vision tasks, such as image denoising, segmentation, restoration and superresolution, object or face detection and recognition in visual scenes, and action recognition. In this manuscript, we provide a brief introduction of the basic theory underlying sparse representation and compressive sensing, and then discuss some methods for recovering sparse solutions to optimization problems in effective way, together with some applications of sparse recovery in a machine learning problem known as sparse dictionary learning.

Sparse Activity and Sparse Connectivity in Supervised Learning

An Overview of Computational Sparse Models and Their Applications in Artificial Intelligence

SparseFormer: Sparse Visual Recognition via Limited Latent Tokens

Exploiting the Full Capacity of Deep Neural Networks while Avoiding Overfitting by Targeted Sparsity Regularization

Integrating Convolution and Sparse Coding for Learning Low-Dimensional Discriminative Image Representations

Sparse Models for Machine Learning

Sparsity-aware generalization theory for deep neural networks

Learning Sparse Latent Representations for Generator Model

Always-Sparse Training by Growing Connections with Guided Stochastic Exploration

Sparsity Invariant CNNs

Network Topology Inference with Sparsity and Laplacian Constraints

Does the $\ell_1$-norm Learn a Sparse Graph under Laplacian Constrained Graphical Models?

SPARLING: Learning Latent Representations with Extremely Sparse Activations

Training a neural netwok for data reduction and better generalization

SparseNet: A Sparse DenseNet for Image Classification

Understanding the Landscape of Sparse Networks: A Brief Investigation

Sparsity Winning Twice: Better Robust Generalization from More Efficient Training

Efficient Marginalization of Discrete and Structured Latent Variables via Sparsity

A Theoretical Explanation of Activation Sparsity Through Flat Minima and Adversarial Robustness

Revisiting Sparse Convolutional Model for Visual Recognition

Projection neural network for a class of sparse regression problems with cardinality penalty