Abstract:In astronomy, we frequently face the decision problem: does this data contain a signal? Typically, a statistical approach is used, which requires a threshold. The choice of threshold presents a common challenge in settings where signals and noise must be delineated, but their distributions overlap. Gravitational-wave astronomy, which has gone from the first discovery to catalogues of hundreds of events in less than a decade, presents a fascinating case study. For signals from colliding compact objects, the field has evolved from a frequentist to a Bayesian methodology. However, the issue of choosing a threshold and validating noise contamination in a catalogue persists. Confusion and debate often arise due to the misapplication of statistical concepts, the complicated nature of the detection statistics, and the inclusion of astrophysical background models. We introduce Conformal Prediction (CP), a framework developed in Machine Learning to provide distribution-free uncertainty quantification to point predictors. We show that CP can be viewed as an extension of the traditional statistical frameworks whereby thresholds are calibrated such that the uncertainty intervals are statistically rigorous and the error rate can be validated. Moreover, we discuss how CP offers a framework to optimally build a meta-pipeline combining the outputs from multiple independent searches. We introduce CP with a toy cosmic-ray detector, which captures the salient features of most astrophysical search problems and allows us to demonstrate the features of CP in a simple context. We then apply the approach to a recent gravitational-wave Mock Data Challenge using multiple search algorithms for compact binary coalescence signals in interferometric gravitational-wave data. Finally, we conclude with a discussion on the future potential of the method for gravitational-wave astronomy.

Monte-Carlo and Bayesian techniques in gravitational wave burst data analysis

Searching for gravitational wave burst in PTA data with piecewise linear functions

Data analysis strategies for the detection of gravitational waves in non-Gaussian noise

On Generation of magnetic field in astrophysical bodies

Bayesian power spectral estimation of gravitational wave detector noise revisited

Gravitational-Wave Detection using Multivariate Analysis

Fast wavelet basis search for generic gravitational wave bursts in Pulsar Timing Array data

Burst search method based on likelihood ratio in Poisson statistics

Posterior predictive checking for gravitational-wave detection with pulsar timing arrays: I. The optimal statistic

Simulation-based inference for stochastic gravitational wave background data analysis

Calibrating gravitational-wave search algorithms with conformal prediction

Gravitational-Wave Data Analysis. Formalism and Sample Applications: The Gaussian Case

Calibrating approximate Bayesian credible intervals of gravitational-wave parameters

An introduction to Bayesian inference in gravitational-wave astronomy: Parameter estimation, model selection, and hierarchical models

Optimizing the low-latency localization of gravitational waves

Posterior predictive checking for gravitational-wave detection with pulsar timing arrays: II. Posterior predictive distributions and pseudo Bayes factors

Enhancing search pipelines for short gravitational wave transients with Gaussian mixture modelling

When is a gravitational-wave signal stochastic?

Detecting the Undetected: Overcoming Biases in Gravitational-Wave Population Studies