Abstract:We study the estimation of distributional parameters when samples are shown only if they fall in some unknown set $S \subseteq \mathbb{R}^d$. Kontonis, Tzamos, and Zampetakis (FOCS'19) gave a $d^{\mathrm{poly}(1/\varepsilon)}$ time algorithm for finding $\varepsilon$-accurate parameters for the special case of Gaussian distributions with diagonal covariance matrix. Recently, Diakonikolas, Kane, Pittas, and Zarifis (COLT'24) showed that this exponential dependence on $1/\varepsilon$ is necessary even when $S$ belongs to some well-behaved classes. These works leave the following open problems which we address in this work: Can we estimate the parameters of any Gaussian or even extend beyond Gaussians? Can we design $\mathrm{poly}(d/\varepsilon)$ time algorithms when $S$ is a simple set such as a halfspace? We make progress on both of these questions by providing the following results: 1. Toward the first question, we give a $d^{\mathrm{poly}(\ell/\varepsilon)}$ time algorithm for any exponential family that satisfies some structural assumptions and any unknown set $S$ that is $\varepsilon$-approximable by degree-$\ell$ polynomials. This result has two important applications: 1a) The first algorithm for estimating arbitrary Gaussian distributions from samples truncated to an unknown $S$; and 1b) The first algorithm for linear regression with unknown truncation and Gaussian features. 2. To address the second question, we provide an algorithm with runtime $\mathrm{poly}(d/\varepsilon)$ that works for a set of exponential families (containing all Gaussians) when $S$ is a halfspace or an axis-aligned rectangle. Along the way, we develop tools that may be of independent interest, including, a reduction from PAC learning with positive and unlabeled samples to PAC learning with positive and negative samples that is robust to certain covariate shifts.

Determining classes for generalized $ψ$-estimators

Comparison and equality of generalized $ψ$-estimators

Monotone representation and measurability of generalized $ψ$-estimators

Axiomatic characterisation of generalized $ψ$-estimators

Basic properties of generalized $ψ$-estimators

Schatten classes of generalized Hilbert operators

Padé-Z2 estimator of determinants

Determination of All Unknown Pure Quantum States with Two Observables

Operator means of probability measures and generalized Karcher equations

On randomized estimators of the Hafnian of a nonnegative matrix

Generalized quantum asymptotic equipartition

Asymptotic $ψ$-densities of subsets of natural numbers

The dimension of the set of $ψ$-badly approximable points in all ambient dimensions; on a question of Beresnevich and Velani

A unified framework for information-theoretic generalization bounds

Efficient Statistics With Unknown Truncation, Polynomial Time Algorithms, Beyond Gaussians

Explicit bound for the number of primes in arithmetic progressions assuming the Generalized Riemann Hypothesis

A Generalization of Relative Entropy to Count Vectors and its Concentration Property

A remark on the set of exactly approximable vectors in the simultaneous case

Private Estimation with Public Data

On quantitative Schur and Dunford-Pettis properties

A Comment on: "A Modern Gauss–Markov Theorem"