Towards a Theory of Pragmatic Information

Edward D. Weinberger
2024-10-05
Abstract:Standard information theory says nothing about how much meaning is conveyed by a message. We fill this gap with a rigorously justifiable, quantitative definition of ``pragmatic information'', the amount of meaning in a message relevant to a particular decision. We posit that such a message updates a random variable, $\omega$, that informs the decision. The pragmatic information of a single message is then defined as the Kulbach-Leibler divergence between the apriori and aposteriori probabilities of $\omega$; the pragmatic information of a message ensemble is the expected value of the pragmatic information of the ensemble's component messages. We justify these definitions by proving that the pragmatic information of a single message is the expected difference between the shortest binary encoding of $\omega$ under the a priori and a posteriori distributions, and that the average of the pragmatic values of individual messages, when sampled a large number of times from the ensemble, approaches its expected value. Pragmatic information is non-negative and additive for independent decisions and ``pragmatically independent'' messages. Also, pragmatic information is the information analogue of free energy: just as free energy quantifies the part of a system's total energy available to do useful work, so pragmatic information quantifies the information actually used in making a decision. We sketch 3 applications: the single play of a slot machine, a.k.a. a ``one armed bandit'', with an unknown payout probability; a characterization of the rate of biological evolution in the so-called ``quasi-species'' model; and a reformulation of the efficient market hypothesis of finance. We note the importance of the computational capacity of the receiver in each case.
Information Theory
What problem does this paper attempt to address?