Abstract:Our article described an experiment that adjudicates between different causal accounts of Bell inequality violations by a comparison of their predictive power, finding that certain types of models that are structurally radical but parametrically conservative, of which a class of superdeterministic models are an example, overfit the data relative to models that are structurally conservative but parametrically radical in the sense of endorsing an intrinsically quantum generalization of the framework of causal modelling. In their comment (<a class="link-https" data-arxiv-id="2206.10619" href="https://arxiv.org/abs/2206.10619">arXiv:2206.10619</a>), Hance and Hossenfelder argue that we have misrepresented the purpose of superdeterministic models. We here dispute this claim by recalling the different classes of superdeterministic models we defined in our article and our conclusions regarding which of these are disfavoured by our experimental results. Their confusion on this point seems to have arisen in part from the fact that we characterized superdeterministic models within a causal modelling framework and from the fact that we referred to this framework as "classical" in order to contrast it with an intrinsically quantum alternative. In this reply, therefore, we take the opportunity to clarify these points. They also claim that if one is adjudicating between a pair of models, where one model can account for strictly more operational statistics than the other, the first model will tend to overfit the data relative to the second. Because this model inclusion relation can arise for pairs of models in a reductionist heirarchy, they conclude that overfitting should not be taken as evidence against the first model. We point out here that, contrary to this claim, one does not expect overfitting to arise generically in cases of model inclusion, so that it is indeed sometimes appropriate to consider overfitting as a criterion for adjudicating between such models.

Comment on "Statistical Modeling: The Two Cultures" by Leo Breiman

Comment on Breiman's "two Cultures" (2002): from Two Cultures to Multicultural

Breiman's "Two Cultures" Revisited and Reconciled

Statistical Modeling: The Three Cultures

Breiman's two cultures: You don't have to choose sides

Modeling, dependence, classification, united statistical science, many cultures

New and notable: Revisiting the "two cultures" through extrinsic noise

The "given data" paradigm undermines both cultures

Considerations Across Three Cultures: Parametric Regressions, Interpretable Algorithms, and Complex Algorithms

Bridging Breiman's Brook: From Algorithmic Modeling to Statistical Learning

Modeling society with statistical mechanics: an application to cultural contact and immigration

Reply to "Comment on 'Experimentally adjudicating between different causal accounts of Bell-inequality violations via statistical model selection'"

Where the Model Frequently Meets the Road: Combining Statistical, Formal, and Case Study Methods

Statistical Thinking, Machine Learning

A biologist's guide to model selection and causal inference

Statistics and Causal Inference

Remarks on the analysis of causal relationships in population research

Comment: Microarrays, Empirical Bayes and the Two-Groups Model

How statistical model development can obscure inequities in STEM student outcomes

Reconciling Heterogeneous Effects in Causal Inference

Model Talk: Calculative Cultures in Quantitative Finance