Abstract:The sequential hypothesis testing problem is a class of statistical analyses where the sample size is not fixed in advance. Instead, the decision-process takes in new observations sequentially to make real-time decisions for testing an alternative hypothesis against a null hypothesis until some stopping criterion is satisfied. In many common applications of sequential hypothesis testing, the data can be highly sensitive and may require privacy protection; for example, sequential hypothesis testing is used in clinical trials, where doctors sequentially collect data from patients and must determine when to stop recruiting patients and whether the treatment is effective. The field of differential privacy has been developed to offer data analysis tools with strong privacy guarantees, and has been commonly applied to machine learning and statistical tasks. In this work, we study the sequential hypothesis testing problem under a slight variant of differential privacy, known as Renyi differential privacy. We present a new private algorithm based on Wald's Sequential Probability Ratio Test (SPRT) that also gives strong theoretical privacy guarantees. We provide theoretical analysis on statistical performance measured by Type I and Type II error as well as the expected sample size. We also empirically validate our theoretical results on several synthetic databases, showing that our algorithms also perform well in practice. Unlike previous work in private hypothesis testing that focused only on the classical fixed sample setting, our results in the sequential setting allow a conclusion to be reached much earlier, and thus saving the cost of collecting additional samples.

Hypothesis Testing Interpretations and Renyi Differential Privacy

Differentially Private Hypothesis Testing, Revisited.

A Statistical Viewpoint on Differential Privacy: Hypothesis Testing, Representation and Blackwell's Theorem

Differentially Private Chi-Squared Hypothesis Testing: Goodness of Fit and Independence Testing

Rényi Differential Privacy

Revisiting Differentially Private Hypothesis Tests for Categorical Data

Statistical Inference in the Differential Privacy Model

Private Sequential Hypothesis Testing for Statisticians: Privacy, Error Rates, and Sample Size

Hyperparameter Tuning with Renyi Differential Privacy

Differentially private Bayesian tests

Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds

On the Statistical Complexity of Estimation and Testing under Privacy Constraints

Differential Privacy: An Economic Method for Choosing Epsilon

General Inferential Limits Under Differential and Pufferfish Privacy

Statistical Approximating Distributions under Differential Privacy.

Differentially Private Kolmogorov-Smirnov-Type Tests

Partial sensitivity analysis in differential privacy

A Statistical Framework for Differential Privacy

Proving differential privacy in Hoare logic

Differential Privacy: on the trade-off between Utility and Information Leakage

Locally Private Hypothesis Testing