Abstract:We study the problem of estimating the parameters of a Boolean product distribution in $d$ dimensions, when the samples are truncated by a set $S \subset \{0, 1\}^d$ accessible through a membership oracle. This is the first time that the computational and statistical complexity of learning from truncated samples is considered in a discrete setting. We introduce a natural notion of fatness of the truncation set $S$, under which truncated samples reveal enough information about the true distribution. We show that if the truncation set is sufficiently fat, samples from the true distribution can be generated from truncated samples. A stunning consequence is that virtually any statistical task (e.g., learning in total variation distance, parameter estimation, uniformity or identity testing) that can be performed efficiently for Boolean product distributions, can also be performed from truncated samples, with a small increase in sample complexity. We generalize our approach to ranking distributions over $d$ alternatives, where we show how fatness implies efficient parameter estimation of Mallows models from truncated samples. Exploring the limits of learning discrete models from truncated samples, we identify three natural conditions that are necessary for efficient identifiability: (i) the truncation set $S$ should be rich enough; (ii) $S$ should be accessible through membership queries; and (iii) the truncation by $S$ should leave enough randomness in all directions. By carefully adapting the Stochastic Gradient Descent approach of (Daskalakis et al., FOCS 2018), we show that these conditions are also sufficient for efficient learning of truncated Boolean product distributions.

Empirical Bayes Estimation of the Truncation Parameter with Asymmetric Loss Function Using NA Samples

Quantile Regression Based on the Weighted Approach with Dependent Truncated Data

Maximum Full Likelihood Approach to Randomly Truncated Data

Maximum likelihood estimation for a special exponential family under random double-truncation

Em Algorithm of the Truncated Multinormal Distribution with Linear Restriction on the Variables

Model Uncertainty and Selection of Risk Models for Left-Truncated and Right-Censored Loss Data

A Non-Parametric Shrinkage Mean Estimation for Arbitrary Quadratic Loss Functions and Unknown Covariance Matrices

Empirical Bayes estimators for the reproduction parameter of Borel-Tanner distribution

Probability Estimation with Truncated Inverse Binomial Sampling

Testing for an ignorable sampling bias under random double truncation

Efficient Parameter Estimation of Truncated Boolean Product Distributions

Non-parametric Shrinkage Mean Estimation for Quadratic Loss Functions with Unknown Covariance Matrices

Variable Selection in Doubly Truncated Regression

Efficient Empirical Bayes prediction under check loss using Asymptotic Risk Estimates

Regression Analysis of Doubly Truncated Data

Addressing the Impact of Data Truncation and Parameter Uncertainty on Operational Risk Estimates

ASYMPTOTICALLY EFFICIENT ESTIMATION IN CENSORED AND TRUNCATED REGRESSION-MODELS

Improved Variance Estimation From Trimmed Samples

A New Approach of Point Estimation from Truncated or Grouped and Censored Data

Estimating Density Models with Truncation Boundaries using Score Matching

Efficient estimation for left-truncated competing risks regression for case-cohort studies