Abstract:The concept of probabilistic values, such as Beta Shapley values and weighted Banzhaf values, has gained recent attention in applications like feature attribution and data valuation. However, exact computation of these values is often exponentially expensive, necessitating approximation techniques. Prior research has shown that the choice of probabilistic values significantly impacts downstream performance, with no universally superior option. Consequently, one may have to approximate multiple candidates and select the best-performing one. Although there have been many efforts to develop efficient estimators, none are intended to approximate all probabilistic values both simultaneously and efficiently. In this work, we embark on the first exploration of achieving this goal. Adhering to the principle of maximum sample reuse, we propose a one-sample-fits-all framework parameterized by a sampling vector to approximate intermediate terms that can be converted to any probabilistic value without amplifying scalars. Leveraging the concept of $ (\epsilon, \delta) $-approximation, we theoretically identify a key formula that effectively determines the convergence rate of our framework. By optimizing the sampling vector using this formula, we obtain i) a one-for-all estimator that achieves the currently best time complexity for all probabilistic values on average, and ii) a faster generic estimator with the sampling vector optimally tuned for each probabilistic value. Particularly, our one-for-all estimator achieves the fastest convergence rate on Beta Shapley values, including the well-known Shapley value, both theoretically and empirically. Finally, we establish a connection between probabilistic values and the least square regression used in (regularized) datamodels, showing that our one-for-all estimator can solve a family of datamodels simultaneously.

Importance Sampling in Bayesian Networks: An Influence-Based Approximation Strategy for Importance Functions

Importance Sampling of Interval Markov Chains

Bayesian Update with Importance Sampling: Required Sample Size

Refractor Importance Sampling

Understanding Approximation for Bayesian Inference in Neural Networks

Using Autodiff to Estimate Posterior Moments, Marginals and Samples

Iterative importance sampling with Markov chain Monte Carlo sampling in robust Bayesian analysis

An automatic adaptive method to combine summary statistics in approximate Bayesian computation

A General Framework for Importance Sampling with Latent Markov Processes

Improving Importance Sampling Method in Structural Reliability

Estimation in hidden Markov models via efficient importance sampling

An Importance Sampling Method with Applications to Rare Event Probability

Implicitly Adaptive Importance Sampling

Importance Sampled Stochastic Optimization for Variational Inference

Fully Bayesian estimation under informative sampling

Efficient Importance Sampling for Rare Event Simulation with Applications

One Sample Fits All: Approximating All Probabilistic Values Simultaneously and Efficiently

Efficient Bayes Inference in Neural Networks through Adaptive Importance Sampling

On a Metropolis-Hastings importance sampling estimator

Evaluating Predictors' Relative Importance Using Bayes Factors in Regression Models.

Deep importance sampling using tensor trains with application to a priori and a posteriori rare event estimation