Abstract:Machine learning researchers and practitioners steadily enlarge the multitude of successful learning models. They achieve this through in-depth theoretical analyses and experiential heuristics. However, there is no known general-purpose procedure for rigorously evaluating whether newly proposed models indeed successfully learn from data. We show that such a procedure cannot exist. For PAC binary classification, uniform and universal online learning, and exact learning through teacher-learner interactions, learnability is in general undecidable, both in the sense of independence of the axioms in a formal system and in the sense of uncomputability. Our proofs proceed via computable constructions that encode the consistency problem for formal systems and the halting problem for Turing machines into whether certain function classes are trivial/finite or highly complex, which we then relate to whether these classes are learnable via established characterizations of learnability through complexity measures. Our work shows that undecidability appears in the theoretical foundations of artificial intelligence: There is no one-size-fits-all algorithm for deciding whether a machine learning model can be successful. We cannot in general automatize the process of assessing new learning models.

The General and Finite Satisfiability Problems for PCTL are Undecidable

The satisfiability problem for a quantitative fragment of PCTL

HyperLTL Satisfiability Is Highly Undecidable, HyperCTL* is Even Harder

Finite Undecidability in PAC and PRC Fields

Some variants of SAT and their properties

Model-Checking PCTL Properties of Stateless Probabilistic Pushdown Systems

Submodel Enumeration for CTL Is Hard

From Undecidability of Non-Triviality and Finiteness to Undecidability of Learnability

On the complexity of model checking interval-valued discrete time Markov chains

Satisfiability of ATL with strategy contexts

Bounded Model Checking of CTL

Satisfiability of Compositional Separation Logic with Tree Predicates and Data Constraints.

On the Satisfiability of Indexed Linear Temporal Logics.

A proof complexity conjecture and the Incompleteness theorem

The Complexity of HyperQPTL

$Π_{2}^{P}$ vs PSpace Dichotomy for the Quantified Constraint Satisfaction Problem

The Connectivity of Boolean Satisfiability: Dichotomies for Formulas and Circuits

Constraint Automata on Infinite Data Trees: From CTL(Z)/CTL*(Z) To Decision Procedures

Decidable verification of uninterpreted programs

On the decidability and complexity of Metric Temporal Logic over finite words

Finite Model Theory and Proof Complexity revisited: Distinguishing graphs in Choiceless Polynomial Time and the Extended Polynomial Calculus