Abstract:Abstract Earthquake forecasting models represent our current understanding of the physics and statistics that govern earthquake occurrence processes. Providing such forecasts as falsifiable statements can help us assess a model’s hypothesis to be, at the least, a plausible conjecture to explain the observations. Prospective testing (i.e., with future data, once the model and experiment have been fully specified) is fundamental in science because it enables confronting a model with completely out-of-sample data and zero degrees of freedom. Testing can also help inform decisions regarding the selection of models, data types, or procedures in practical applications, such as Probabilistic Seismic Hazard Analysis. In 2010, a 10-year earthquake forecasting experiment began in Italy, where researchers collectively agreed on authoritative data sources, testing rules, and formats to independently evaluate a collection of forecasting models. Here, we test these models with ten years of fully prospective data using a multiscore approach to (1) identify the model features that correlate with data-consistent or -inconsistent forecasts; (2) evaluate the stability of the experiment results over time; and (3) quantify the models’ limitations to generate spatial forecasts consistent with earthquake clustering. As each testing metric analyzes only limited properties of a forecast, the proposed synoptic analysis using multiple scores allows drawing more robust conclusions. Our results show that the best-performing models use catalogs that span over 100 yr and incorporate fault information, demonstrating and quantifying the value of these data types. Model rankings are stable over time, suggesting that a 10-year period in Italy can provide sufficient data to discriminate between optimal and suboptimal forecasts. Finally, no model can adequately describe spatial clustering, but those including fault information are less inconsistent with the observations. Prospective testing assesses relevant assumptions and hypotheses of earthquake processes truly out-of-sample, thus guiding model development and decision-making to improve society’s earthquake resilience.

Earthquake forecasting and its verification

High Resolution Long- and Short-Term Earthquake Forecasts for California

Optimization of the Approach to Systematic Earthquake Forecasting

Earthquake Predictability and Forecast Evaluation Using Likelihood‐Based Marginal and Conditional Scores

Earthquake forecasting from paleoseismic records

Evaluation of a Decade-Long Prospective Earthquake Forecasting Experiment in Italy

Residual analysis methods for space--time point processes with applications to earthquake forecast models in California

Evaluating the impact of uncertainty in ground motion forecasts for post-earthquake impact modeling applications

Testing earthquake predictions

Applying the Akaike Information Criterion (AIC) in earthquake spatial forecasting: a case study on probabilistic seismic hazard function (PSHF) estimation in the Sumatra subduction zone

Statistical earthquake focal mechanism forecasts

Optimizing Earthquake Nowcasting With Machine Learning: The Role of Strain Hardening in the Earthquake Cycle

Do earthquakes "know" how big they will be? a neural-net aided study

Ranking earthquake forecasts using proper scoring rules: Binary events in a low probability environment

A Bayesian Approach for Forecasting the Probability of Large Earthquakes Using Thermal Anomalies from Satellite Observations

Magnitude Uncertainties Impact Seismic Rate Estimates, Forecasts and Predictability Experiments

Exploring the Link Between Seismic and Atmospheric Parameters Using Spatio Temporal Analysis: Implications for Earthquake Forecasting

Performance of On-Site Earthquake Early Warning System Using Strong-Motion Records from Recent Earthquakes

The theoretical and practical foundations of strong earthquake predictability

Embracing Data Incompleteness for Better Earthquake Forecasting

Implementing Non-Poissonian Forecasts of Distributed Seismicity into the 2022 Aotearoa New Zealand National Seismic Hazard Model