A Note on the Self-Consistency of Definitions of Generalization and Inductive Inference
H. Kramer
DOI: https://doi.org/10.1145/321119.321125
1962-04-01
Abstract:One of the characteristic and sometimes annoying propensities of the human intellect is that of inferring general rules from limited experience. For some purposes, it is of interest to cause hardware to imitate this distinctly soft behavior. Unfortunately, computing machinery is not amenable to the customary ways of human persuasion and, prior to teaehing computing machines to generalize , it is necessary first to specify what is meant by generalization with sufficient precision to permit translation of the definition into a computer program. A class of definitions that holds promise in this regard is based on the following idea. A finite sequence of instances E1, E2, ... , E,~ is presented to the eomputer possibly together with some conditional statements. The job of generalization or inductive inference consists of specifying a class of objects whieh includes the given finite sequence of examples and satisfies the side conditions. In the present context, the class is considered to be specified if the computer has a routine for generating any member of the class. That is to say, the class is defined by a program for a function f on the integers for which f(1) = E1 , f(2) = E~, ..., f(n) = E~ and which satisfies all of the given constraints. Let ~Y(E1, E2, •-, E~) be the class of functions which satisfy the above conditions and thus may be considered potential generalizations. Which of these functions should be chosen as the generalization? R. J. Solomonoff [1] has suggested that the function in ~y(E~, E2,- .-, E,,) be selected , whose program contains the smallest number of bits. Actually, this is just one of several possible definitions of complexity and we shall suppose more generally that the choice is based merely on some numerical index of the functions that we shall deal with (cognoscenti might try their luck with a partial ordering). The question arises whether any such definition of generalization is selfeon-sistent. Suppose a sequence of events E,, E2, ... , E,, is given, and the generalization f* is found. Consider now the sequence of events El, E2, ... , E~ f*(n + 1). Is it true that f* is also a generalization for this expanded sequence of instances? The answer is "yes". It will be seen that the demonstration of this fact is very straightforward and does not depend on any specific notion of complexity. Let us restrict our …