Abstract:Statistical tests of earthquake predictions require a null hypothesis to model occasional chance successes. To define and quantify `chance success' is knotty. Some null hypotheses ascribe chance to the Earth: Seismicity is modeled as random. The null distribution of the number of successful predictions -- or any other test statistic -- is taken to be its distribution when the fixed set of predictions is applied to random seismicity. Such tests tacitly assume that the predictions do not depend on the observed seismicity. Conditioning on the predictions in this way sets a low hurdle for statistical significance. Consider this scheme: When an earthquake of magnitude 5.5 or greater occurs anywhere in the world, predict that an earthquake at least as large will occur within 21 days and within an epicentral distance of 50 km. We apply this rule to the Harvard centroid-moment-tensor (CMT) catalog for 2000--2004 to generate a set of predictions. The null hypothesis is that earthquake times are exchangeable conditional on their magnitudes and locations and on the predictions--a common ``nonparametric'' assumption in the literature. We generate random seismicity by permuting the times of events in the CMT catalog. We consider an event successfully predicted only if (i) it is predicted and (ii) there is no larger event within 50 km in the previous 21 days. The $P$-value for the observed success rate is $<0.001$: The method successfully predicts about 5% of earthquakes, far better than `chance,' because the predictor exploits the clustering of earthquakes -- occasional foreshocks -- which the null hypothesis lacks. Rather than condition on the predictions and use a stochastic model for seismicity, it is preferable to treat the observed seismicity as fixed, and to compare the success rate of the predictions to the success rate of simple-minded predictions like those just described. If the proffered predictions do no better than a simple scheme, they have little value.

Statistical tests for evaluating an earthquake prediction method

Testing earthquake predictions

Earthquake Predictability and Forecast Evaluation Using Likelihood‐Based Marginal and Conditional Scores

Non-parametric Tests for the Tail Equivalence Via Empirical Likelihood

Earthquake forecasting and its verification

Earthquake Number Forecasts Testing

On the Testing of Seismicity Models

Residual analysis methods for space--time point processes with applications to earthquake forecast models in California

Magnitude Uncertainties Impact Seismic Rate Estimates, Forecasts and Predictability Experiments

Reliability Of Seismic Resistance Predictions

Posterior predictive checking for gravitational-wave detection with pulsar timing arrays: I. The optimal statistic

Assessment of Point Process Models for Earthquake Forecasting

Statistical Failure Mechanism Analysis of Earthquakes Revealing Time Relationships

Embracing Data Incompleteness for Better Earthquake Forecasting

Deviation from power law of the global earthquake seismic moment distribution

Search for Reliable Precursors: A Case Study of the Seismic Quiescence of the 2000 Western Tottori Prefecture Earthquake

Ranking earthquake forecasts using proper scoring rules: Binary events in a low probability environment

High Resolution Long- and Short-Term Earthquake Forecasts for California

Correlation‐based tests of predictability

Bayesian earthquake forecasting approach based on the epidemic type aftershock sequence model

Statistical significance testing for mixed priors: a combined Bayesian and frequentist analysis