Abstract:The study of intelligent systems explains behaviour in terms of economic rationality. This results in an optimization principle involving a function or utility, which states that the system will evolve until the configuration of maximum utility is achieved. Recently, this theory has incorporated constraints, i.e., the optimum is achieved when the utility is maximized while respecting some information-processing constraints. This is reminiscent of thermodynamic systems. As such, the study of intelligent systems has benefited from the tools of thermodynamics. The first aim of this thesis is to clarify the applicability of these results in the study of intelligent systems. We can think of the local transition steps in thermodynamic or intelligent systems as being driven by uncertainty. In fact, the transitions in both systems can be described in terms of majorization. Hence, real-valued uncertainty measures like Shannon entropy are simply a proxy for their more involved behaviour. More in general, real-valued functions are fundamental to study optimization and complexity in the order-theoretic approach to several topics, including economics, thermodynamics, and quantum mechanics. The second aim of this thesis is to improve on this classification. The basic similarity between thermodynamic and intelligent systems is based on an uncertainty notion expressed by a preorder. We can also think of the transitions in the steps of a computational process as a decision-making procedure. In fact, by adding some requirements on the considered order structures, we can build an abstract model of uncertainty reduction that allows to incorporate computability, that is, to distinguish the objects that can be constructed by following a finite set of instructions from those that cannot. The third aim of this thesis is to clarify the requirements on the order structure that allow such a framework.

Order Parameters for Minimax Entropy Distributions: when Does High Level Knowledge Help?

Order Parameters for Detecting Target Curves in Images: When Does High Level Knowledge Help?

Learning Bayesian Network Parameters Via Minimax Algorithm

Learning Bayesian Network Parameters from Limited Data by Integrating Entropy and Monotonicity

Learning Bayesian Networks Using the Constrained Maximum a Posteriori Probability Method

Atr Applications Of Minimax Entropy Models Of Texture And Shape

Learning Bayesian Network Parameters from Small Data Set: A Spatially Maximum a Posteriori Method.

The Principle of Uncertain Maximum Entropy

Criterion of Significance Level for Selection of Order of Spectral Estimation of Entropy Maximum

Maximum Entropy Principle, Equal Probability a Priori and Gibbs Paradox

Second-Order Characterization of Minimax Parameter Estimation in Restricted Parameter Space

Entropic Priors and Bayesian Model Selection

Should We Learn Most Likely Functions or Parameters?

Generalised Entropy MDPs and Minimax Regret

The Maximum Likelihood Estimation with the Order Statistics

Optimal Observables for Minimum-Error State Discrimination in General Probabilistic Theories

Approximate information maximization for bandit games

Order-theoretic models for decision-making: Learning, optimization, complexity and computation

Estimators of Entropy and Information via Inference in Probabilistic Models

Marginally Specified Priors for Nonparametric Bayesian Estimation

Minimax-Optimal Bounds for Detectors Based on Estimated Prior Probabilities