Abstract:Approximation of high-dimensional functions is a problem in many scientific fields that is only feasible if advantageous structural properties, such as sparsity in a given basis, can be exploited. A relevant tool for analysing sparse approximations is Stechkin's lemma. In its standard form, however, this lemma does not allow to explain convergence rates for a wide range of relevant function classes. This work presents a new weighted version of Stechkin's lemma that improves the best $n$-term rates for weighted $\ell^p$-spaces and associated function classes such as Sobolev or Besov spaces. For the class of holomorphic functions, which occur as solutions of common high-dimensional parameter-dependent PDEs, we recover exponential rates that are not directly obtainable with Stechkin's lemma. Since weighted $\ell^p$-summability induces weighted sparsity, compressed sensing algorithms can be used to approximate the associated functions. To break the curse of dimensionality, which these algorithms suffer, we recall that sparse approximations can be encoded efficiently using tensor networks with sparse component tensors. We also demonstrate that weighted $\ell^p$-summability induces low ranks, which motivates a second tensor train format with low ranks and a single weighted sparse core. We present new alternating algorithms for best $n$-term approximation in both formats. To analyse the sample complexity for the new model classes, we derive a novel result of independent interest that allows the transfer of the restricted isometry property from one set to another sufficiently close set. Although they lead up to the analysis of our final model class, our contributions on weighted Stechkin and the restricted isometry property are of independent interest and can be read independently.

Weighted SPICE: A Unifying Approach for Hyperparameter-Free Sparse Estimation

A Note on the SPICE Method

Weighted LASSO for Sparse Recovery with Statistical Prior Support Information

Generalized Sparse Covariance-based Estimation

Weighted sparsity and sparse tensor networks for least squares approximation

Efficient Probabilistic Latent Semantic Analysis with Sparsity Control

On the Conditions of Sparse Parameter Estimation via Log-Sum Penalty Regularization.

Hyperparameter Estimation for Sparse Bayesian Learning Models

Diffusion Sparse Least-Mean Squares over Networks

Weighted Least-Squares PARSIM

Weighted Lasso Estimates for Sparse Logistic Regression: Non-Asymptotic Properties with Measurement Errors

Fast Laplace Approximation for Sparse Bayesian Spike and Slab Models

Sparse Poisson Regression with Penalized Weighted Score Function

Training Sparse Neural Network by Constraining Synaptic Weight on Unit Lp Sphere

Nonparametric augmented probability weighting with sparsity

Sparse Density Estimation with Measurement Errors

On Sparse Estimation for Semiparametric Linear Transformation Models

Iterative-Weighted Thresholding Method for Group-Sparsity-Constrained Optimization With Applications

Sparse Principal Component Analysis via Joint L2,1-Norm Penalty

Estimating Sparse Signals Using Integrated Wideband Dictionaries