Abstract:Artificial intelligence has made great strides since the deep learning revolution, but AI systems still struggle to extrapolate outside of their training data and adapt to new situations. For inspiration we look to the domain of science, where scientists have been able to develop theories which show remarkable ability to extrapolate and sometimes predict the existence of phenomena which have never been observed before. According to David Deutsch, this type of extrapolation, which he calls "reach", is due to scientific theories being hard to vary. In this work we investigate Deutsch's hard-to-vary principle and how it relates to more formalized principles in deep learning such as the bias-variance trade-off and Occam's razor. We distinguish internal variability, how much a model/theory can be varied internally while still yielding the same predictions, with external variability, which is how much a model must be varied to accurately predict new, out-of-distribution data. We discuss how to measure internal variability using the size of the Rashomon set and how to measure external variability using Kolmogorov complexity. We explore what role hard-to-vary explanations play in intelligence by looking at the human brain and distinguish two learning systems in the brain. The first system operates similar to deep learning and likely underlies most of perception and motor control while the second is a more creative system capable of generating hard-to-vary explanations of the world. We argue that figuring out how replicate this second system, which is capable of generating hard-to-vary explanations, is a key challenge which needs to be solved in order to realize artificial general intelligence. We make contact with the framework of Popperian epistemology which rejects induction and asserts that knowledge generation is an evolutionary process which proceeds through conjecture and refutation.

Reintroducing Prediction to Explanation

On the Relationship Between Explanation and Prediction: A Causal View

Prediction can be safely used as a proxy for explanation in causally consistent Bayesian generalized linear models

The prediction-explanation fallacy: A pervasive problem in scientific applications of machine learning

Predictions as statements and decisions

Prediction in the Hypothesis-Rich Regime

You Can Do Better! if You Elaborate the Reason when Making Prediction.

Evaluating computational models of explanation using human judgments

Applying Deutsch's concept of good explanations to artificial intelligence and neuroscience -- an initial exploration

Building Better Theories: Prediction Intervals as a Tool for Theory Testing and Improvement

Psychological Theory and the Illusion of Scientific Prediction

Explanation, prediction, and conceptual exploration

Semantic Explanations of Predictions

Prediction and Inference: From Models and Data to Artificial Intelligence

Prediction in reading: A review of predictability effects, their theoretical implications, and beyond

Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction

CUBE: Causal Intervention-Based Counterfactual Explanation for Prediction Models

The Automated Laplacean Demon: How ML Challenges Our Views on Prediction and Explanation

Explaining Predictive Uncertainty by Looking Back at Model Explanations

Theory Before the Test: How to Build High-Verisimilitude Explanatory Theories in Psychological Science